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SECTION  1 

INTRODUCTION  AND  SUMMARY 


INTRODUCTION 

Interest  in  multisensor  combined  displays  has  arisen  from  a  variety 
of  different  sources.  First,  since  cockpit  space  is  severely  limit¬ 
ed,  any  way  of  reducing  space  for  displays  while  still  maintaining 
the  required  information  has  considerable  appeal.  Secondly,  the 
pilot/operator  workload  is  constantly  increasing  as  mission  pro¬ 
files  for  high-performance  aircraft  become  more  complex  and  demand¬ 
ing.  Multisensor  combined  displays  provide  the  potential  for  both 
conserving  cockpit  space  and,  at  the  same  time,  reducing  the  op¬ 
erator's  workload  by  consolidating  and  integrating  information 
from  two  different  sources  (sensors)  onto  one  display  surface. 

Advances  in  display  technology  and  image  processing  techniques 
have  recently  made  such  a  concept  feasible.  Given  that  it  is  elec¬ 
tronically  possible  to  combine  on  one  display  information  from 
two  or  more  sensors,  the  question  then  becomes  one  of  how  such 
information  can  be  meaningfully  integrated.  For  example,  if 
one  were  to  consider  combining  forward-looking  infra-red  ( FLIR ) 
imagery  with  radar  imagery  on  one  display  surface,  a  number  of 
questions  and  problems  immediately  arise.  The  range,  resolution, 
and  field  of  view  differences  between  the  two  sensors,  for  ex¬ 
ample,  suggest  that  a  simple  superimposition  of  the  two  sensor 
outputs  would  create  interpretation  problems  for  the  operator. 

Since  the  goal  is  to  reduce  workload  and  enhance  performance,  the 
entire  process  of  combining  information  must  be  carefully  analyzed 
and  implemented. 
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This  report  covers  the  second  phase  of  a  three-phase  program 
sponsored  by  the  Office  of  Naval  Research.  Phase  I  (ONR  Contract 
Number  N00014-76-C-0797)  addressed  feasibility  issues.  Various 
methods  of  combining  sensor  information  were  discussed  and  pre¬ 
liminary  evaluations  made.  Phase  II,  was  concerned  with  definition 
and  the  preliminary  design  of  several  combined  display  concepts. 
Phase  III  was  an  evaluation  of  these  Phase  II  concepts  in  terms 
of  operator  performance. 

To  define  the  techniques  most  suitable  for  combining  multiple  sen¬ 
sors  into  one  displayed  image,  it  was  necessary  in  Phase  II  to 
limit  the  scope  by  performfing  the  following  analytical  tasks: 

1)  Mission  scenarios  were  developed  using  the  Navy's 
current  and  projected  attack  aircraft  as  the 
sensor  platform.  The  emphasis  was  on  defining  the 
operator's  information  requirements  during  critical 
mission  phases. 

2)  FLIR  and  radar  were  selected  as  the  primary  sensors 
and  their  characteristics  defined.  Additional  sen¬ 
sors  may  be  added  and,  in  some  instances,  Low  Light 
Level  Television  (LLLTV)  substituted  for  the  FLIR 
sensor . 

3)  Several  different  concepts  for  combining  specified 
sensor  information  were  selected.  These  concepts 
made  use  of  both  color  and  black-and-white  displays. 
Photographic  examples  illustrating  these  concepts 
were  prepared. 
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4)  A  test  plan  was  prepared  for  evaluating  these 

combined  display  concepts.  The  plan  involved  both 
analytical  and  experimental  evaluations.  The 
experimental  portion  realistically  simulated  the 
combined  displays  in  a  dynamic  scenario.  Operator 
target  acquisition  was  one  basis  for  comparison. 
The  actual  test  was  conducted  during  Phase  III. 

During  Phase  III  the  following  tasks  were  performed: 

1)  Background  imagery  for  the  stimulus  material 
was  collected  by  flying  a  helicopter  over  a 
predescribed  path  in  a  rugged  and  uninhabited 
area  along  Minnesota's  North  Shore. 

2)  Software  was  developed  to  produce  the  simulated 
combined  and  uncombined  display  conditions. 

3)  The  image  processing  facility  consisting  of  a 
Honeywell  Level  6  computer  and  a  Stanford  Tech¬ 
nology  Model  70  imagery  system  was  used  to  pro¬ 
duce  identical  5-minute  segments  of  multisensor 
color  combined  display,  multisensor  black  and 
white  combined  display,  and  uncombined  individu¬ 
al  IR  and  PPI  radar  display. 

4)  The  Man  Computer  Laboratory  was  used  to  conduct 
the  experiment  which  included  a  point -of-impact 
task,  a  target  recognition  task  and  a  secondary 
workload  task.  Target  recognition  used  a  voice 
recognition  system. 
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5)  The  data  was  analyzed  using  statistical  techniques 
to  determine  the  advantages  and  problems  of  com¬ 
bined  displays. 

6)  Recommendations  were  generated  to  aid  the  configu¬ 
ration  of  combined  multisensor  displays.  An  advanced 
combined  multisensor  concept  for  high-speed  pene¬ 
tration  and  pop-up  was  developed. 

The  product  of  each  of  the  above  tasks  is  described  in  detail  in 
the  remainder  of  this  report . 

SUMMARY 

The  analytical  and  experimental  data  studied  under  this  effort 
indicates  that  combined  multisensor  displays  may  offer  signifi¬ 
cant  improvements  in  mission  profiles  in  the  following  ways: 

•  Improving  navigation 

•  Permitting  lower  altitude  operation 

•  Increasing  the  number  of  targets  detected 
and  recognized 

•  Reducing  time  to  detect  and  recognize  targets 

•  Improved  weapon  delivery 

•  Reduce  pilot  workload 

In  conclusion,  combined  multisensor  displays  provide  a  high-lever 
age  methodology  (significant  improvements  with  a  minimum  invest¬ 
ment)  to  improve  the  pilot/aircraft  interface  which  in  turn  pro¬ 
duces  significant  improvements  in  the  mission  profile. 


SECTION  2 


DEVELOPMENT  OF  A  MISSION  SCENARIO 


The  analysis  of  representative  missions  within  the  context  of  this 
study  had  a  two-fold  purpose: 

1)  To  determine  where  a  multisensor  combined 
display  might  be  beneficially  employed 

2)  To  provide  a  framework  for  evaluating  the 
selected  combining  concepts  in  Phase  III 

Meeting  the  first  objective  required  a  detailed  look  at  where  and 
how  the  various  sensors  are  currently  being  used  in  typical  combat 
missions.  It  was  necessary  to  determine  the  operator's  specific 
tasks  and  information  requirements  during  the  course  of  a  variety 
of  missions.  Under  which  circumstances  were  multiple  sensor  inputs 
used?  What  types  of  information  was  the  operator  attempting  to 
locate  or  integrate  across  the  several  sensor  inputs?  An  under¬ 
standing  of  the  operator's  information  requirements  eased  the 
understanding  of  how  the  combined  display  could  erihance  informa¬ 
tion  extraction.  A  variety  of  missions  were  reviewed,  as  discussed 
below. 


MISSIONS 

The  multisensor  concepts  developed  during  this  effort  must  bene¬ 
fit  the  projected  mission  profiles.  A  review  of  both  operational 
and  functional  time  sequences  will  assist  in  determining  the 
strengths  and  weaknesses  of  the  current  multisensor  concepts. 


The  most  useful  data  sources  on  the  tactical  aircraft  missions 
are  the  test  material,  and  the  staff  at  the  Fighter  Weapons  School 
at  Nellis  AFB.  This  information  includes  the  Flight  Manuals; 

Flight  Crew  Checklists  and  Performance  Data  Manuals  for  current 
aircraft;  and  Technical  Orders  for  the  aircraft  systems.  All  of 
these  sources  were  used  to  prepare  the  mission  profiles  and  re¬ 
quirements.  The  most  current  definitions  of  air-to-ground  missions 
and  operations  for  tactical  aircraft  are  reflected  herein.  The 
basic  source  of  the  post-1985  mission  requirements  is  the  TAC-851 
study.  Because  it  was  conducted  by  USAF  Headquarters  in  1970,  it 
needed  a  review  and  some  updating. 


Mission  Model 

The  mission  model  describes  the  operational  requirements  and  per¬ 
formance  characteristics  for  tactical  aircraft  operating  after 
1985.  The  primary  operational  environment  is  Central  Europe  during 
a  major  NATO/Warsaw  Pact  engagement.  The  performance  descriptions 
emphasize  the  areas  which  will  influence  aircrew  functions  and 
interface  with  the  aircraft. 


Mission  Requirements 

Mission  requirements  are  defined  for  an  aircraft  performing  deep 
strike,  battlefield  interdiction  and  close  air  support  missions. 

The  general  characteristics  of  the  mission  are  compared  in  Table  1. 
Mission  profiles  for  each  mission  are  described. 

"Air  Force  Tactical  Forces  1985  Study  (U),"  Final  Report,  HQ  TAC , 
Langley  AFB,  Virginia,  May,  1971,  Secret. 
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Deep-Strike  Mission  Profile 


Deep-strike  missions  attack  airfields,  surveillance  and  GCI 
radars,  Command , Control  and  Communication  facilities,  and  high- 
value  transportation  and  supply  targets.  There  are  several  essen¬ 
tial  operational  characteristics  and  requirements  for  these  mis¬ 
sions  : 


•  Targets  are  at  a  known  geographical  position 

•  Imagery  for  the  targets  is  available 

•  Targets  are  fixed  or  will  remain  in  place  for  a 
period  of  days 

•  Threats  to  the  mission  have  a  significant  impact 
on  flight  profiles  and  on  the  allowable  time  with¬ 
in  the  threat  zones  of  operation. This  is  reflected 
in  altitude  and  air-speed  profiles,  and  in  the 
operation  of  threat -warning  and  threat-avoidance 
systems 

•  Weather  conditions  include  a  low  cloud  cover  and 
limited  visibility.  During  the  winter,  the  aver¬ 
age  visibility  is  low  and  the  average  cloud 
ceilings  are  2000  feet 

•  High  sortie  rates  are  required  to  rapidly  re¬ 
duce  the  effectiveness  of  enemy  forces.  This,  in 
turn,  imposes  the  need  for  an  all-weather  and 
day/night  capability 


The  deep-strike  mission  profile  is  shown  as  a  series  of  seg¬ 
mented  functions  in  Figure  1.  The  individual  segments  and  the 
primary  functions  are  described  in  the  figure. 
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MISSION  PHASE 

FUNCTIONS 

A 

PRE  TAKE-OFF 

A/C  EQUIPMENT  STARTS,  TEST  AND 
CHECKOUT,  REFUEL  AND  RELOAD 

B 

TAKE-OFF  AND  ACCELERATE  TO 
OPTIMUM  CLIMB 

A/C  CONTROL,  VISUAL  OBSTRUCTION, 
TRAFFIC  COORDINATION 

C 

CLIMB  TO  CRUISE  ALTITUDE- 
USE  30,000  FT. 

A/C  CONTROL,  NAVIGATION, 

TRAFFIC  COORDINATION 

D 

RENDE2C0US  WITH  OTHER 

AIRCRAFT  ONMISSION 

A/C  CONTROL,  VISUAL/SENSOR 

SEARCH,  MISSION  COORDINATION 

£ 

CRUISE  AT  M  0.9  FOR  200  nmi 

A/C  CONTROL,  N  A  VIGATION, 

FORMATION  FLYING,  THREAT 

SURVEILLANCE 

F 

CLIMB  TO  DASH  ALTITUDE: 

60,000  FT. 

A/C  CONTROL,  NAVIGATION, 

FORMATION  FLYING,  THREAT 

SURVEILLANCE 

G 

HIGH  SPEED  INGRESS  AT 

M  =  2.0  FOR  200  nmi 

A/C  CONTROL,  NAVIGATION, 

FORMATION  FLYING,  THREAT 

SURVEILLANCE 

H 

TARGET  SEARCH  USING  SAR 

A/C  CONTROL,  NAVIGATION, 

COORD. ATTACK, THREAT 

SURVEILLANCE,  SAR  DATA 

PROCESSING,  TARGET  IDENTIFICATION 

AND  TRACK 

1 

WEAPON  DELIVERY  USING 

BOOST/GLIDE  ROCKET 

A/C  CONTROL,  DESIGNATE  TARGET. 

LAUNCH  ROCKET,  THREAT  EVALUATION, 
THREAT  DESIGNATION,  LAUNCH 

ARMAMENT,  TARGET  TRACK  30MB  DAMAGE 
ASSESSMENT,  COORDINATE  ATTACK 

J 

HIGH  SPEED  WITHDRAWAL  AT 

M  2.0  FOR  200  nmi 

A/C  CONTROL,  NAVIGATION,  THREAT 
SURVEILLANCE,  FORMATION  FLYING 

K 

DESCEND  TO  CRUISE  ALTITUDE 

A/C  CONTROL,  NAVIGATION,  THREAT 
SURVEILLANCE,  FORMATION  FLYING 

L 

CRUISE  AT  M  0.9  FOR  200  nmi 

A/C  CONTROL,  NAVIGATION,  THREAT 
SURVEILLANCE,  FORMATION  FLYING 

M 

DESCEND  TO  AIRFIELD 

A/C  CONTROL,  NAVIGATION,  TRAFFIC 
COORDINATION 

N 

LAND 

A/C  CONTROL,  VISUAL  OBSTRUCTION, 

TRAFFIC  COORDINATION,  LANDING 

AID  OPERATION 

Figure  1.  Deep  Strike  Mission  Profile 
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The  mission  profile  is  based  on  the  following  assumptions: 


•  The  aircraft  is  part  of  a  coordinated  mission 
using  several  aircraft 

•  Synthetic  aperture  radar  is  used  as  a  long-range 
ground  search  sensor 

•  Synthetic  aperture  retransmission  guidance  (SARG) 
missile  is  used  for  the  target  attack 

A  flight  profile  for  the  mission  is  shown  in  Figure  2.  A  time  pro¬ 
file  for  the  mission  is  shown  in  Figure  3. 

Alternative  sensor  and  weapon  combinations  for  this  mission  are 
as  follows: 

•  GPS-controlled  launch  point  (GBU-15  weapon  and 
FLIR  sensor  with  data  link) 

•  Inertial  navigation-controlled  launch  point  (SOM 
with  TERCOM) 


Battlefield  Interdiction  Mission  Profile 

Battlefield  interdiction  missions  are  against  weapons,  personnel, 
transportation  and  supplies  of  the  enemy's  ground  forces.  The 
essential  operational  characteristics  and  requirements  for  this 
mission  are  as  follows: 

•  Targets  are  identified  by  aerial  reconnaissance 
and  surveillance  sensors  and  are  at  a  known 
location 
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Figure  2.  Deep  Strike  Mission  Flight  Profile 
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Figure  3. 


Deep  Strike  Mission  Time  Profile 
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•  There  is  no  imagery  for  the  target 

•  Targets  are  mobile  or  highly  transient 

•  Weather  and  viewing  conditions  limit  the  useful¬ 
ness  of  visual  operations.  Day/night  and  all- 
weather  sensors  and  weapons  are  needed  to  meet 
the  requirements  for  many  of  the  sorties 

•  High  sortie  rates  are  required  to  deal  with  the 
large  number  of  targets 

•  Time  of  flight  to  the  target  area  must  be  mini¬ 
mized  because  of  the  transience  of  the  target 

•  Threat  to  the  mission  includes  the  tactical  SAMS 
and  AA  Guns  that  can  operate  against  the  very 
low  altitude,  high-speed  mission  profile.  This 
includes  SA6 ,  SA8 ,  and  ZSU-23-4 

The  battlefield  interdiction  mission  profile,  is  shown  as  a  series 
of  segments  in  Figure  4.  Individual  segments  are  described  and 
the  primary  functions  listed  in  the  figure.  The  mission  profile 
is  based  on  the  following  assumptions: 

•  The  aircraft  is  part  of  a  coordinated  mission 
using  several  aircraft 

•  Forward-looking  infrared  (FLIR)  is  used  for  a 
ground  search  sensor 

•  Imaging  IR  (IIR)  Maverick  is  used  for  attacking 
the  target 
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MISSION  PHASE 

FUNCTIONS 

A 

PRE  TAKE-OFF 

A/C  EQUIPMENT  STARTS,  TEST  AND 

CHECKOUT,  REFUEL  AND  RELOAD 

B 

TAKE-OFF  AND  ACCELERATE  TO 
CLIMB 

A/C  CONTROL,  VISUAL  OVSERVATION, 

TRAFFIC  COORDINATION 

C 

CLIMB  TO  CRUISE  ALTITUDE— 

30K  FT 

A/C  CONTROL,  NAVIGATION,  TRAFFIC 
COORDINATION 

D 

RENDEZVOUS  WITH  OTHER 

AIRCRACT  ON  MISSION 

A/C  CONTROL,  VISUAL/SENSOR  SEARCH, 

MISSION  COORDINATION 

E 

CRUISE  AT  M  0.9  FOR  200  nmi 

A/C  CONTROL,  NAVIGATION,  FORMATION 

FLYING,  THREAT  SURVEILLANCE 

F 

RENDEZVOUS  WITH  TANKER 

A/C  CONTROL,  VISUAL/SENSOR  SEARCH, 

MISSION  COORDINATION,  THREAT 

SURVEILLANCE 

G 

REFUEL 

A/C  CONTROL,  FUEL  TRANSFER,  MISSION 
COORDINATION,  THREAT  SURVEILLANCE 

H 

DESCEND,  LOITER  AT  MINIMUM 
POWER  FOR  15  MIN 

A/C  CONTROL,  NAVIGATION,  MISSION 
COORDINATION,  THREAT  SURVEILLANCE 

1 

HIGH  SPEED  PENETRATION  M  1.2 

AT  100  FT  ALTITUDE  FOR  50  nmi 

A/C  CONTROL,  NAVIGATION,  MISSION 
COORDINATION,  TERRAIN  AVOIDANCE, 

THREAT  SURVEILLANCE,  ECM  OPERATION, 
VISUAL/SENSOR  SEARCH 

J 

TARGET  SEARCH  USING  FUR, 

POP-UP  TO  SEARCH  ALTITUDE 

A/C  CONTROL.  FUR  SEARCH,  TARGET 
IDENTIFICATION  AND  TRACK,  THREAT 
SURVEILLANCE,  ECM  AND  DECOY  OPERATIONS, 
MISSION  COORDINATION 

K 

WEAPON  DELIVERY  USING  HR 
MAVERICK 

A/C  CONTROL,  WEAPON  LOCK-ON,  WEAPON 
LAUNCH,  THREAT  SURVEILLANCE.  ECM  AND 

DECOY  OPERATIONS,  MISSION  COORDINATION, 
DAMAGE  ASSESSMENT 

L 

HIGH  SPEED  WITHDRAWAL 

SAME  AS  1 

M 

CLIMB  TO  CRUISE  ALTITUDE 

SAME  AS  H 

N 

CRUISE  TO  BASE  AREA 

SAME  AS  E 

0 

DESCEND  TO  AIRFIELD 

SAME  AS  C 

P 

LAND 

A/C  CONTROL,  VISUAL/SENSOR  SEARCH, 

TRAFFIC  COORDINATION,  LANDING 

AID  OPERATION 

_ 

Figure  4.  Battlefield  Interdiction 
Mission  Profile 
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A  flight  profile  for  the  mission  is  presented  in  Figure  5,  and 
time  profile  is  presented  in  Figure  6. 


Alternative  sensors  and  weapons  for  this  mission  include: 

•  Visual  identification  of  the  target;  CBU  delivery 
using  CCIP  fire  control  in  a  laydown  mode 

•  SLAR  identification  of  the  target;  with  GBU-15 
delivery  of  minimissile  submunitions 


Close-Air-Support  Mission  Profile 

Close  air  support  missions  are  conducted  at  the  request  of  the 
tactical  ground  forces'  commander.  A  ground-  or  air-based  for¬ 
ward  air  controller  coordinates  the  air  attack  with  the  ground 
forces  by  designating  the  target  position  and  by  controlling  the 
air-support  mission  during  the  attack  phase.  The  targets  include 
enemy  weapons,  troops,  fortifications,  and  transports  that  are 
an  immediate  threat  to  the  mission  of  the  ground  forces. 

The  essential  operational  characteristics  and  requirements  for 
this  mission  are  as  follows: 

•  Targets  are  generally  mobile,  and  may  be  moving 

•  Targets  have  been  located  by  the  tactical  ground 
forces  and  are  identified  by  position  or  by  a 
designator  operated  by  the  forward  air  controller 

•  An  ability  to  strike  the  target  on  a  few  minutes 
notice  is  needed 
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TIME  (MINTUES) 


igure  b.  Battlefield  Interdiction  Mission  Time  Pro 


•  Threat  to  the  mission  will  be  primarily  the  SA-8, 

SA-9  and  ZSU-23-4  weapons  that  are  operating  within 
the  attacking  units 

•  The  aircraft  must  be  able  to  remain  on  airborne 
station  and  to  make  a  series  of  target  attacks 
from  the  station  as  needed 

•  Weather  conditions  will  include  low  clouds  and 
limited  visibility 

The  close-air-support  mission  profile  is  shown  as  a  series  of 
segments  in  Figure  7.  The  individual  segments  and  the  primary 
functions  are  described  in  the  figure. 

The  mission  profile  is  based  on  the  following  assumptions: 

•  The  aircraft  is  part  of  a  coordinated  mission 
using  several  aircraft 

•  The  target  is  identified  by  the  FAC  using  a 
laser  designator 

•  Laser  Maverick  is  used  to  attack  the  target 

A  typical  flight  profile  for  the  mission  is  shown  in  Figure  8.  A 
time  profile  is  shown  in  Figure  9. 

Alternative  sensor  and  weapon  combinations  are  as  follows: 

•  Visual  identification  of  the  target;  30-mm  gun 
a  tack 
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MISSION  PHASE 

FUNCTIONS 

A 

PRE  TAKE-OFF 

A/C  EQUIPMENT  STARTS,  TEST  AND  CHECKOUT, 
REFUEL  AND  RELOAD 

B 

TAKE-OFF  AND  ACCELERATE  TO  CLIMB 

A/C  CONTROL,  VISUAL  OBSERVATION, 

TRAFFIC  COORDINATION 

C 

CLIMB  TO  CRUISE  ALTITUDE— 25K  ft 

A/C  CONTROL,  NAVIGATION,  TRAFFIC 

COORDINATION 

D 

RENDEZVOUS  WITH  OTHER  AIRCRAFT 

ON  MISSION 

A/C  CONTROL,  VISUAL/SENSOR  SEARCH, 

MISSION  COORDINATION 

E 

CRUISE  AT  M  0.7  FOR  100  r?mi 

A/C  CONTROL,  NAVIGATION,  FORMATION 

FLYING,  THREAT  SURVEILLANCE 

F 

DESCEND  AND  LOITER  AT  MINIMUM 

POWER  FOR  IS  MIN 

A/C  CONTROL,  NAVIGATION,  FORMATION  FLYING, 
THREAT  SURVEILLANCE,  COORDINATION  WITH  FAC 

G 

PENETRATE  TO  FEBA 

A/C  CONTROL,  NAVIGATION,  FORMATION  FLYING, 
THREAT  SURVEILLANCE,  COORDINATION  WITH  FAC 

H 

TARGET  SEARCH  LASER  DETECTOR 

A/C  CONTROL,  THREAT  SURVEILLANCE,  LASER 

SEARCH  AND  TRACK,  COORDINATION  WITH  FAC, 

ECM  OPERATION 

1 

WEAPON  DELIVERY  USING  LASER 
MAVERICK 

A/C  CONTROL,  WEAPON  LOCK-ON  AND  LAUNCH, 

THREAT  SURVEILLANCE,  ECM  OPERATION 

J 

WITHDRAWAL  TO  LOITER  POSITION, 
REPEAT  ATTACK  PROCESS  AS  NEEDED 

SAME  AS  F 

K 

CUMB  to  cruise  altitude 

SAME  AS  F 

L 

CRUISE  TO  BASE  AREA 

SAME  AS  E 

M 

DESCEND  TO  AIRFIELD 

SAME  AS  C 

N 

LAND  , , 

A/C  CONTROL,  VISUAL/SENSOR  TRAFFIC 
COORDINATION,  LANDING  AND  OPERATION 

Figure  7.  Close-Air  Support  Mission  Profile 
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TIME  (MINUTES) 


Figure  9.  Close-Air  Support  Mission  Time  Profile 


•  Launch  relative  to  FAC  controlled  beacons;  iner- 
tially  guided  GBU-15  with  sensor  fuzed  submunitions 

Threats 

To  support  the  need  for  high-speed,  low-level  intrusion's  flights, 
Figure  10  shows  the  typical  radar  coverage  of  three  missile  sites 
for  three  aircraft  altitudes.  The  immediate  site  is  a  system  set 
up  within  5  minutes  of  a  roadway.  The  expedient  site  is  within 
30  minutes  of  a  roadway  and  the  pre-planned  site  was  selected 
for  its  excellent  coverage.  The  light  areas  represent  the  seg¬ 
ments  where  the  aircraft  is  exposed  to  tracking  by  the  radar 
site.  Slow  flying  aircraft  at  1000  feet  over  a  preplanned  site 
may  experience  a  50  times  increase  in  radar  tracking  exposure 
as  compared  to  high-speed  aircraft  at  250  feet  over  immediate 
sites . 

WEAPONS 

This  section  reviews  a  wide  range  of  air-to-ground  weapons  which 
may  interact  with  multisensor  displays.  The  actual  selection  of 
weapons  in  subsequent  tasks  will  depend  on  the  type  of  target  and 
its  value,  the  threat  to  the  aircraft,  and  the  weather  conditions. 
Information  used  to  define  the  operational  functions  for  the  cur¬ 
rent  weapons  and  the  aircrew's  interface  with  them  has  been  ob¬ 
tained  from  flight  manuals,  equipment  tech  orders  and  from  weapons 
specifications.  Information  on  the  advanced  weapons  was  obtained 
from  reports  on  design  studies  on  specific  weapons  and  from  the 
TAC-85  study.  The  weapons  appropriate  for  after  1985  include  most 
of  the  current  inventory  of  weapons,  and  many  weapons  that  are  in 
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Figure  10.  Intervisibility  Map  for  Threat  Sites 
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various  phases  of  development.  The  current  inventory  includes  a 
wide  variety  of  unguided  bombs  and  rockets,  the  Pave  Way  and  Pave 
Strike  program  developments,  the  Maverick  family,  the  ARM  weapons,, 
and  gun  systems.  Programs  in  the  developmental  process  include 
the  stand-off  missile  (SOM)  and  improved  ARM,  improved  guidance 
using  radio-navigation  techniques  and  sensor  data-correlation 
techniques;  and  all-weather  sensor/trackers . 


Weapons  Categorization 

A  broad  selection  of  weapons  was  made  from  this  total  so  that  most 
of  the  possibilities  are  represented.  The  general  characteristics 
of  the  initial  selection  of  weapons  are  outlined  in  Table  2.  The 
weapon  interface  with  the  aircraft  and  crew  is  outlined  in  Table 
3.  The  weapons  are  organized  in  functional  groups  having  similar 
characteristics  in  terms  of  the  aircrew  and  aircraft  interfaces: 

•  Unguided  Weapons.  These  weapons  require  the 
aircrew  to  visually  acquire  the  target  and  to 
use  the  aircraft  fire  control  system  to  obtain 
the  correct  conditions  for  launching  weapons 

•  Lock-On-Before-Launch  Weapons .  These  weapons 
require  the  aircrew  to  visually,  or  through 
sensors,  acquire  the  target;  to  lock  the  weapon 
seeker  onto  the  target;  and  to  launch  the 
weapon 
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TABLE  2.  INITIAL  WEAPON  SELECTION 
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TABLE  3.  PRELIMINARY  WEAPON/ AIRCRAFT/ CREW  INTERFACE 
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•  Lock-On-After-Launch  Weapons .  These  weapons 
require  the  aircrew  to  navigate  to  a  known 
position  and  to  launch  the  weapon  toward  the 
target;  to  control  the  weapon  seeker  so  as 
to  obtain  the  target  area  imagery  via  the 
data  link;  to  identify  the  target  in  the 
imagery  and  to  control  the  seeker  to  lock  onto 
the  target 

•  Autonomous  Weapons .  These  weapons  require  the 
aircrew  to  navigate  to  a  position  relative  to 
the  target  and  to  launch  the  weapon  toward  the 
target . 


Unguided  Weapons 

The  unguided  weapons  are  the  general  choice  when  the  anti-aircraft 
threat  is  small.  Using  the  advanced  bombing  systems-CCIP  (Continu¬ 
ously  Computed  Impact  Point)  and  ARBS  (Angle  Rate  Bombing  System) 
very  good  weapon  delivery  accuracy  is  obtained. 

With  release  at  short  range,  and  using  an  appropriate  warhead, 
good  effectiveness  is  obtained  against  most  of  the  tactical  tar¬ 
gets.  The  unguided  weapons  listed  in  Tables  2  and  3  included  the 
following : 

•  General-purpose  bombs  —  Effective  against  all  of 
the  tactical  targets  except  the  small,  hard  targets 
which  require  a  direct  hit 
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•  Cluster  bomb  units  —  Alternative  subraunitions 
and  effective  against  the  total  range  of  tacti¬ 
cal  targets.  Both  area  targets  and  point  targets 
can  be  attacked  by  controlling  the  submunition 
pattern.  Sensor-fuzed  busmunitions  are  good 
against  concentrations  of  armor 

•  Fuel  air  explosives  —  Fuel-air-explosive  (FAE) 
devices  are  particularly  effective  against  area 
targets  that  are  vulnerable  to  overpressure 

•  Snakeye  —  This  is  representative  of  the  retarded 
weapons  ..designed  for  launching  at  very  low  alti¬ 
tudes 

•  30-mm  Cannon  —  This  weapon  represents  the  short- 
range,  boosted  category,  which  includes  both 
rockets  and  cannons.  The  cannon  is  effective 
against  the  small,  hard  target  when  used  at  short 
range.  The  development  of  a  flexible  gun  will 
improve  the  attack  options 

External  factors  influence  the  effectiveness  of  the  aircrew: 
search  area,  search  time  and  conspicuousness  of  the  target.  In 
general,  the  aircrew  should  have  a  minimum  of  distractions  when 
conducting  a  search;  the  transition  from  search  to  attack  should 
require  a  minimum  of  time;  and  the  pilot  should  have  no  distrac¬ 
tions  while  delivering  the  weapon. 
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Lock-On-Before-Launch 


The  lock-on-before-launch  weapons  include  Maverick,  HOBO,  and  one 
of  the  GBU-15  configurations.  These  weapons  have  been  used  very 
effectively  in  Vietnam,  primarily  to  destroy  bridges;  and  in  the 
Arab-Israeli  conflicts  to  destory  tanks.  They  provide  both  a 
longer  stand-off  and  a  better  terminal  accuracy  than  the  unguided 
weapons . 

They  require  the  aircrew  to  visually  acquire  the  target  by  ground 
search  and/or  by  search  of  sensor  video  data.  The  target  must  be 
designated  in  the  sensor  video  such  that  the  weapon  sensor  will 
start  tracking  the  target.  With  weapon  tracking  established,  it 
can  be  launched  as  soon  as  the  aircraft  is  within  the  launch  en¬ 
velope. 

External  factors  also  influence  the  aircrew's  effectiveness: 
Search  area,  search  time,  and  conspicuousness  of  the  target.  In 
general,  the  aircrew  should  have  a  minimum  of  distractions  during 
the  target  search  operation;  and  the  transition  from  search  to 
seeker  lock-on  should  occur  in  as  little  time  as  possible. 


Lock-On-After-Launch 

The  lock-on-after-launch  weapons  include  Condor  and  one  of  the 
GBU-15  and  SOM  configurations.  These  weapons  are  being  developed 
and  have  a  critical  vulnerability  to  ECM  because  they  need  a  high 
data  rate  (video)  data  link.  They  will  be  very  useful  against 
high-value,  highly  defended,  stationary  (for  several  hours)  tar¬ 
gets  . 


Operating  in  a  relatively  safe  area,  the  aircrew  navigates  to  a 
preselected  launch  point  and  maneuvers  to  align  the  weapon's  iner¬ 
tial  system  and  to  obtain  the  needed  launch  conditions.  As  the 
weapon  approaches  check  points,  or  approaches  the  target  area,  the 
data  link  must  be  activated  to  control  the  weapon's  sensor  search 
and  to  obtain  the  sensed  data.  The  check  point  or  target  is  iden¬ 
tified  in  the  displayed  sensor  data  and  is  designated  to  the  wea¬ 
pon  sensor.  Depending  on  the  stage  of  the  mission,  either  the 
navigation  errors  are  corrected,  or  terminal  homing  is  initiated. 

The  external  factors  which  influence  the  aircrew’s  effectiveness 
are  weather,  targeting  accuracy,  target  conspicuousness,  and  data 
link  link  countermeasures.  The  weapon-system  factors  are  search 
data,  search  time,  and  sensor  definition  of  the  ground  scene. 

As  general  guidelines,  the  aircrew  should  have  a  minimum  of  dis¬ 
tractions  during  the  target  search,  and  the  transition  from  search 
to  lock-on,  to  terminal  flight  should  require  a  minimum  amount  of 
time . 

Autonomous  Weapons 

The  autonomous  weapons  have  a  mix  of  functions,  including  defense 
suppression,  stand-off  deep  strikes,  and  close  air  support.  In 
common,  they  relieve  the  aircrew  of  searching  for  the  target.  The 
weapons  include  most  of  the  GBU-15  and  SOM  configurations,  the 
laser-guided  weapons  and  the  ARM  weapons . 

The  laser-guided  weapons  require  that  the  target  be  illuminated 
by  a  laser  designator.  This  requires  a  search  and  acquisition  pro¬ 
cess,  but  will  be  conducted  by  an  FAC  or  other  aircraft  in  the 
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mission.  Both  the  laser-guided  weapon  and  the  ARM  weapon  may  detect 
the  target  prior  to  launching  and  may  provide  information  for  fire 
control  and  weapon  launch.  The  GBU-15  and  SOM  require  that  the 
aircrew  navigate  to  a  launching  position  in  a  safe  area.  Maneuvers 
then  occur  to  align  the  weapon's  inertial  navigation  systems  and 
to  obtain  the  desired  launching  conditions. 


Conclusions  of  Mission  and  Weapon  Analyses 

A  preliminary  evaluation  of  mission  profiles  and  weapon  systems 
suggested  that  the  current  multisensor  concepts  apply  and  offer 
major  benefits  for  the  battlefield  interdiction  mission,  and  the 
close  air  support  mission .  They  may  also  aid  in  controlling  un¬ 
guided  weapons,  and  some  lock-on-before-launch  weapons.  However, 
the  evaluation  also  indicated  that  the  concepts  may  be  expanded 
to  cover  lock-on-after-launch  systems  containing  a  video  link. 

In  addition,  the  concepts  require  an  expansion  to  cover  three  and 
four  sensors. 
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SECTION  3 


DEFINITION  OF  SENSOR  CHARACTERISTICS 


To  better  understand  the  potential  advantages  of  combined  sensor 
display  and  to  generate  effective  concepts  for  combining  sensor 
inputs  on  a  single  display,  it  was  necessary  to  review  the  charac¬ 
teristics  of  the  individual  sensors.  For  the  particular  aircraft 
and  missions  of  interest ,  FLIR  and  radar  were  the  most  promising 
candidates.  Each  of  these  sensors  operates  in  a  different  mode, 
responds  to  different  properties  within  the  field  of  view,  and 
(in  traditional  displays)  presents  this  information  to  the  opera¬ 
tor  in  different  formats.  The  following  paragraphs  summarize  some 
of  the  more  important  properties  of  these  two  sensors.  The  end  of 
this  section  discusses  the  function  of  a  "second  sensor." 


35-GHz  RADAR 

Detection  and  surveillance  of  ground  targets  require  a  system  of 
high  angular  resolution.  High-resolution  radars  demand  a  small 
beamwidth  or  large  aperture  antennas.  In  a  high-performance  air¬ 
craft,  the  antenna  size  is  usually  limited  to  the  available  areas 
in  the  nose  or  an  externally  attached  radar  pod. 

Since  the  effective  antenna  aperture  is  inversely  proportional  to 
frequency,  millimeter  waves  (f  >  30  GHz)  offer  the  following: 

•  Superior  angular  resolution  for  a  given  antenna 
physical  size 
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•  More  nearly  covert  operation  (low  probability 
of  intercept) 

•  Lower  susceptibility  to  jamming 

Millimeter  wave  (MMW)  radars,  however,  are  susceptible  to  such 
environmental  factors  as  rainfall  attenuation,  which  reduces  the 
overall  effective  range.  These  factors  must  be  carefully  evalu¬ 
ated  to  predict  the  MMW  radar  detection  range. 

As  an  example,  an  environmental  diagram  was  designed  to  illustrate 
the  detection  ranges  of  reasonable  size  antennas  (12",  8",  6")  at 
35  GHz.  This  hypothetical  system  can  readily  be  modified  by  chang¬ 
ing  some  radar  parameters  (such  as  transmitter  power,  target  size, 
etc.)  to  predict  performances  in  clear  weather  and  inclement  wea¬ 
ther.  In  comparing  35  GHz  and  94  GHz,  94  GHz  has  the  clear  advan¬ 
tage  in  angular  resolution,  but  has  a  significantly  higher  rainfall 
attenuation  characteristic. 

Figures  11  and  12  are  read  by  selecting  the  target  size  (cross 
section)  on  the  bottom  scale  and  going  up  to  the  antenna/frequency 
lines.  At  this  intersection,  the  range  is  read  out  in  km  on  the 
right-hand  scale.  To  find  the  range  due  to  weather  attenuation, 
the  range  on  the  right-hand  scale  is  traced  back  across  the  graph 
to  the  appropriate  attenuation  curve  near  the  left.  At  this  inter¬ 
section,  the  range  reduction  due  to  precipitation  attenuation  is 
shown  in  km  on  the  top  scale. 

For  surveillance,  a  MMW  system  could  be  designed  with  a  reasonable 
angular  resolution  (say  about  1-2  degrees)  at  antenna  scan  rates 
up  to  120°/sec.  To  ensure  adequate  surveillance,  the  following 
factors  must  be  considered: 
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REDUCTION  DUE  TO  PRECIPITATION  ATTENUATION 
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Figure  11.  Environmental  Diagram:  Large  Antennae 


Km  SCALE  -  RANGE  REDUCTION  DUE  TO  PRECIPITATION  ATTENUATION 


Figure  12.  Environmental  Diagram:  Small  Antennae 


(1)  Aircraft  velocity  and  altitude 

(2)  Antenna  beamwidth  and  scan  rates 


For  detecting  light  to  moderate  rainfall,  ranges  of  3-7  km  can  be 
achieved  with  the  hypothetical  MMW  radar.  The  radar  outputs  would 
be  azimuth  and  elevation  (relative  to  the  gimbled  platform),  and 
range.  Angular  accuracies  on  the  order  of  1-2  milliradians  and 
range  accuracies  of  about  50  feet  can  be  expected.  The  antenna 
could  be  scanned  in  both  axes,  or  use  a  fixed  depression  angle  and 
scanned  in  azimuth. 


DUAL-MODE  RADAR 

The  three-watt,  35-GHz  radar  system  discussed  has  a  small  antenna, 
high  resolution  and  low  susceptibility  to  jamming.  However,  as 
shown  in  Figure  13,  the  35-GHz  system's  performance  deteriorates 
rapidly  in  poor  weather. 

To  overcome  these  limitations,  a  second,  dual-mode,  air-to-ground 
system  was  considered  for  high-performance  attack  aircraft .  This 
radar,  with  an  IR  and/or  EO  sensor,  provides  the  pilot  with  a 
multispectral  display  of  the  target  area.  For  navigating,  the  radar 
also  displays  terrain  features  extending  above  the  aircraft's  alti¬ 
tude  . 

The  transmitter  is  a  10-watt  pulsed  Impatt  diode  employing  a  spread 
spectrum  centered  at  16  GHz.  The  pulse-width  is  200ns  with  a  25- 
MHz  chirp  bandwidth.  Frequencies  higher  than  16  GHz  are  considered 
impractical  for  this  application  due  to  the  high  rain  clutter  and 
rain  attenuation  encountered  in  the  terrain-avoidance  mode.  Lower 
frequencies  do  not  provide  the  required  angular  resolution. 
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15  GHz  CENTER  FREQ. 

10  WATT  PEAK  POWER 
200  ns  PULSEWIDTH 
25  MHz  PULSE  BANDWIDTH 


Figure  13.  Functional  Block  Diagram  of  the  Dual  Mode 
16-GHz  Radar  System 


In  a  rainfall  of  4  mm/hr,  an  operating  range  of  5  km  is  achievable 
in  the  target -detection  mode,  and  10  km  in  the  terrain-avoidance 
mode.  The  two  modes  are  time-shared,  with  detection  performed  on 
the  left  to  the  right  scan  at  a  1.75  degree  depression  angle,  and 
the  terrain  mapping  on  the  returning  right -to-left  scan  at  a  -1.5 
degree  depression  angle. 

Terrain  avoidance  and  the  detection  of  ground  targets  are  both 
air-to-ground  functions  and  impose  similar  requirements  on  the 
radar  sensor.  Therefore,  a  single  radar  can  accomplish  both  func¬ 
tions.  However,  the  radar  antenna  must  be  tilted  above  the  hori¬ 
zon  to  perform  the  terrain-avoidance  function,  and  below  the  hori¬ 
zon  to  detect  targets.  Thus,  two  time-shared  modes  of  operation 
are  required. 

A  functional  block  diagram  of  the  proposed  design  is  presented  in 
Figure  13.  The  RD  and  IF  (Intermediate  Frequency)  processing  are 
identical  for  the  two  modes,  except  for  the  pulse  repetition  fre¬ 
quency  ( PRF ) ,  which  is  defined  as: 

PRF  =  - 

unamb , 


where 

C  =  speed  of  light  and 

Runamb  ~  Maximum  unambiguous  range  (i.e.,  echoes 

correspond  to  previously  transmitted  pulse). 
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The  5-km  maximum  operating  range  in  the  detection  mode  permits  a 
maximum  PRF  of  30  kHz.  A  15-kHz  PRF  is  employed  in  the  terrain 
avoidance  mode,  which  has  a  10  km  operating  range. 


The  video  processor  has  22  contiguous  range  gates,  each  matched  to 
the  200  ns-transmitted  pulsewidth.  In  the  detection  mode,  these 
gates  are  implemented  to  cover  the  660m  ground  swath  from  4.34  to 
5  km.  The  660m  coverage  is  necessary,  since  the  aircraft  moves 
approximately  this  distance  (800  fps)  during  the  full  2.66-sec 
antenna-scan  period.  This  scan  period  assumes  a  minimum  gimbal 
scan  rate  of  60°/sec  and  a  ±45°  field  of  view.  The  detection  mode 
occurs  on  the  left-to-right  scan  and  the  terrain  avoidance  on  the 
returning  right -to-left  scan.  If  a  wider  ground  coverage  is  de¬ 
sired  for  mapping,  more  range  gates  can  be  implemented.  However, 
as  indicated  in  Figure  14,  the  maximum  range  swath  is  limited  to 
the  interval  from  2.7  to  5  km  due  to  antenna  illumination.  (Note 
that  a  depression  angle  of  1.75  degree  is  required  to  maximize 
antenna  gain  for  targets  at  the  5  km  range. 

In  the  terrain-avoidance  mode,  the  22  gates  are  implemented  to 
cover  the  660m  ground  swath  from  9.34  to  10  km.  As  illustrated  in 
Figure  15,  the  antenna  is  tilted  upwards  at  1.5°.  The  antenna 
gain  seen  by  flat  terrain  at  a  range  of  10  km  is  8  dB  below  the 
mainlobe  gain,  which  is  viewing  terrain  above  the  aircraft's  line 
of  flight.  Therefore,  the  undesired  signal  returns  from  terrain 
below  the  aircraft’s  altitude  will  be  at  least  16  dB  (two-way 
antenna  gain  difference)  lower  than  the  desired  signal  returns 
from  terrain  above  the  aircraft's  altitude.  An  adaptive  threshold 
can  easily  be  implemented  which  will  ensure  that  the  radar  re¬ 
sponds  only  to  the  desired  higher  altitude  terrain  returns.  A 
sensitivity  time  control  (STC)  function  must  also  be  implemented 
to  remove  the  effect  of  range  on  received  signal  level. 
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Figure  14.  Target  Detection  Geometry 


Figure  15.  Terrain  Avoidance  Geometry 


Preliminary  calculations  indicate  that,  at  the  maximum  operational 

ranges,  the  signal-to-noise  ratio  (SNR)  and  the  signal-to-clutter 

ratio  (SCR>  are  both  greater  than  the  required  14  dB  (0.95.  proba- 

—6 

bility  of  detection  and  10“  probability  of  false  alarm. 

FLIR 

The  following  paragraphs  briefly  describe  the  major  characteristics 
of  existing  FLIR  sensors,  and  some  of  the  features  being  consid¬ 
ered  for  the  second-generation  FLIRs,  primarily  the  use  of  image 
processing  for  enhancing  the  displayed  image. 

Current  FLIR  Systems 

Current  state-of-the-art  thermal  imaging  systems  generally  fall 
into  three  categories:  serial  scan,  serial -parallel  scan,  and 
parallel  scan.  All  can  be  characterized  by  the  generic  block  dia¬ 
gram  of  Figure  16. 

Thermal  imagery  search  effectiveness  is  limited  by  deficiencies 
resulting  from 

•  Greater  dynamic  range  in  the  scene  than  can 
usefully  be  displayed 

•  Constraints  on  system  resolution 

The  display's  dynamic  range  constraints  have  been  partially  re¬ 
solved  in  the  past  by  an  a-c  coupling  of  the  detectors  with  a 
subsequent  d-c  restoration. 
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Figure  16.  First  Generation  Thermal  Imager 
Block  Diagram 
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The  serial-scan  approach  uses  a  two-axis  scanning  system  to  scan 
a  detector  or  array  of  detectors  such  that  each  detector  sees  the 
entire  field  of  view.  This  system  has  distinct  advantages:  de¬ 
tector  responsivity  equalization  is  not  necessary,  d-c  restoration 
is  easy,  and  fewer  components  are  used.  The  result  is  a  simpler 
system.  The  disadvantages  are  that  high  bandwidth  detectors  must 
be  used,  and  the  mechanical  scanners  tend  to  be  complex,  which  is 
the  inherent  weak  link  in  the  system.  High  sensitivity  in  serial 
scan  systems  requires  multi-element  arrays  in  place  of  a  single¬ 
detector  element.  The  elements  are  used  in  the  time-delay-and-add 
mode,  which  increases  the  effective  signal-to-noise  ratio  by  the 
square  root  of  the  number  of  detectors  in  the  array. 

Serial-parallel  scan  systems  have  similar  characteristics  to  serial 
scan  systems,  except  that  more  than  one  line  is  scanned  at  a  time. 
The  additional  line  scans  are  delayed  for  insertion  into  the  out¬ 
put  video  at  the  proper  time.  This  technique  allows  the  horizontal 
mirror  to  operate  at  slower  rates,  but  the  delay  and  multiplexing 
electronics  become  more  complex  as  the  number  of  lines  simultane¬ 
ously  scanned  increases. 

Fully  parallel  systems  eliminate  the  need  for  delay  lines.  Current 
parallel  scan  systems  are  either  viewed  directly  or  electro-opti- 
cally  multiplexed  to  provide  TV-compatible  imagery  to  either  an 
eyepiece,  in  the  case  of  direct  view  systems,  or  to  a  vidicon  TV 
camera.  Parallel-scan  systems  have  simpler  scanning  mechanisms 
and  a  greater  sensitivity  because  of  increased  detector  dwell 
time.  Responsivity  equalization  and  d-c  restoration  is  complex  in 
parallel  systems  because  of  the  large  number  of  detectors.  Also, 
if  a  detector  is  lost,  then  one  line  of  information  is  lost.  This 
is  common  in  large  detector  arrays. 
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System  Design  Considerations 


Designing  a  FLIR  for  an  optimum  performance  involves  evaluating 
many  components  and  subsystems.  Most  tradeoffs  depend  on  the  FLIR 
application,  and  different  components  are  usually  selected  speci¬ 
fically  for  that  application  and  for  its  environment.  Examples  of 
application-specific  components  are  the  f ield-of-view  defining 
optics,  the  type  of  detector  cooling,  the  FLIR  stabilization 
scheme,  the  power  supply,  and  the  location  and  size  of  the  display 
There  are  several  considerations,  however,  which  can  be  studied 
in  general  because  an  optimum  selection  can  be  made  that  covers  a 
wide  range  of  applications.  Two  of  the  most  significant  of  these 
considerations  are  the  scanning  method  and  the  electronic  proces¬ 
sing  technique. 


Scanning  Method 


The  most  important  decision  in  designing  a  FLIR  is  selecting  the 
scanning  method,  because  this  affects  choices  of  other  components, 
as  well  as  on  the  overall  system  performance.  Two  basic  types  of 
scanning  are  presently  representative  of  most  FLIRs:  parallel 
scanning,  and  serial  scanning. 


Historically,  the  first  high-performance  FLIRs  used  parallel  scan¬ 
ning.  Since,  in  a  parallel-scanned  system  each  detector  element 
dissects  only  a  small  portion  of  the  FOV  (one  or  two  lines),  rela¬ 
tively  narrow  bandwidths  are  required.  A  serial-scanned  system 
requires  that  each  detector  dissect  the  entire  FOV,  resulting  in 
a  wide  bandwidth.  When  FLIR  development  began,  no  detector  materi¬ 
al  was  available  with  the  short-time  constant  needed  for  these 
wide  bandwidths.  Consequently,  serial-scanned  FLIRs  were  not  de- 


veloped.  The  application  of  short-time  constant  (Hg,  Cd)  Te  de¬ 
tector  arrays  to  thermal  imaging  systems  made  possible  the  re¬ 
quired  wide  bandwidths  for  serial-scanned  FLIRs. 

Most  present  thermal  imagers  use  a-c  coupling.  This  results  in  an 
average  value  of  zero  for  the  video  of  each  channel  in  a  parallel- 
scanned  FLIR,  even  though  the  average  value  of  the  thermal  emis¬ 
sion  from  the  scene  may  vary  from  channel  to  channel.  An  imperfect 
reproduction  of  the  scene  results.  An  example  of  this  effect  is 
the  difficulty  of  a  parallel-scanned  FLIR  in  detecting  horizons. 

For  many  applications,  horizon  detection  is  a  very  desirable,  and 
sometimes  necessary  feature  for  orientation  during  search  and 
acquisition . 

Recent  techniques  using  CCD  and  CID  arrays,  together  with  the 
development  of  photovoltaic  detectors  exhibiting  negligible  1/f 
noise,  subtract  background  noise  and  dynamic  channel  balancing 
for  parallel-scanned  arrays,  thus  eliminating  many  of  these  prob¬ 
lems  . 

In  a  parallel-scanned  system  which  does  not  have  the  complexity  of 
d-c  restoration  to  a  common  thermal  reference,  hot  targets  tend 
to  suppress  the  background  on  the  channels  which  scan  the  hot  tar¬ 
gets.  This  occurs  since  the  average  value  of  the  video  in  the  a-c- 
coupled  electronics  is  zero.  From  line-to-line ,  the  average  value 
of  the  scene  radiance  will  vary.  These  values  are  considerably 
different  when  hot  targets  are  present.  Thus,  background  suppres¬ 
sion  results  on  those  channels  containing  hot  targets.  In  a  serial- 
scanned  FLIR,  the  average  value  of  the  video  represents  the  aver¬ 
age  value  of  the  entire  scene.  Thus,  hot  targets  do  not  suppress 
the  background  on  individual  lines  of  the  display.  Another  hot 
target  effect  is  the  undershoot  resulting  from  inadequate  low- 
frequency  response  of  the  video  processing  electronics. 
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The  serial-scanned  FLIR  has  a  high  scan  rate  with  a  correspond¬ 
ingly  short  dwell  time.  Therefore,  for  a  fixed  low-frequency 
response,  a  serial  scanner  exhibits  less  undershoot  compared  to 
a  parallel-scanned  system  which  has  a  much  longer  detector  dwell 
time.  However,  the  high  scan  rate  in  the  serial-scanned  FLIR 
results  in  a  higher  degree  of  mechanical  complexity  than  parallel 
scanning . 

A  combination  of  parallel  and  serial  scanning  using  time  delay 
and  integration  can  provide  a  higher  performance  at  the  expense 
of  increased  complexity  on  the  focal  plane  if  high-density  area 
detector  arrays  become  feasible.  The  ultimate  in  mechanical  sim¬ 
plicity  is  the  "staring”  array  which  substitutes  electronic  for 
mechanical  scanning  entirely.  However,  we  do  not  feel  that  star¬ 
ing  arrays  will  be  a  candidate  for  a  second-generation  FLIR. 

There  are  several  constraining  tradeoffs  in  designing  a  thermal 
imaging  system.  The  visibility  of  image  features  is  directly 
affected  by  the  contrast  between  features  on  the  display  medium. 
Current  thermal  imagers  offer  some  control  over  contrast  through 
gain  and  brightness  adjustments,  either  by  automatically  sensing 
the  video  data,  or  through  the  operator's  control.  However,  these 
are  global  controls  which  cannot  be  optimized  for  the  operator's 
target  detection,  recognition  and  identification  tasks  on  all 
types  of  imagery.  For  example,  clutter  (the  number  of  "target- 
like"  objects  within  the  field)  is  a  significant  factor  in  search 
effectiveness . 
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Problems  in  achieving  adequate  contrast  sensitivity  with  current 
thermal  imaging  systems  stem  mainly  from  five  factors: 


•  Large  dynamic  range  of  thermal  imagery 

•  Low  contrast  levels  of  certain  targets 

•  Limited  dynamic  range  of  the  electronics 

•  Limited  display  dynamic  range 

•  Operator  workload  (for  manual  gain/brightness 
control ) 

Diurnal  and  seasonal  variations  of  thermal  imagery  can  typically 
range  over  30  degrees  to  40  degrees  for  daily  cycles,  and  up  to 
150  degrees  for  seasonal  cycles.  Such  ranges  of  temperature  indi¬ 
cate  a  need  for  gain/brightness  control,  but  it  need  not  be  auto¬ 
matic  to  compensate  for  these  slowly  varying  extremes. 

FLIR  imagery  usually  contains  a  portion  of  the  horizon  sky  which 
typically  falls  10  degrees  to  40  degrees  below  the  ambient  ground 
temperature  (and  in  some  instances  more  than  100  degrees,  e.g., 
a  cool,  clear  night  over  a  desert).  Within  ground  imagery,  ter¬ 
rain  features  may  vary  in  temperature  by  20  degrees  -  30  degrees, 
while  targets  of  military  interest  can  exhibit  contrast  levels 
ranging  from  a  degree  or  less  (e.g.,  a  camouflaged,  passive  tar¬ 
get)  to  several  thousand  degrees  (e.g.,  a  jet  exhaust).  If,  for 
example,  the  MRT  (minimum  resolvable  temperature)  at  the  IFOV 
(instantaneous  field  of  view)  is  0.1  degree,  the  dynamic  range 
(peak  image  intensity  to  rms  noise  level)  within  the  scene  could 
be  on  the  order  of  10,000:1.  Current  S0A  (state  of  the  art)  in 
CCD  array  processor  dynamic  range  is  typically  an  order  of  mag¬ 
nitude  less  (1000:1).  Even  more  constraining,  typical  thermal 


46 


imagery  displays  have  dynamic  ranges  of  15:1  to  30:1,  while  the 
very  best  quality  monitors  available  have  less  than  100:1  dyna¬ 
mic  range. 

The  design  of  any  thermal  imaging  system  involves  a  classical 
tradeoff  between  sensitivity,  resolution  and  data  rate.  Given  a 
specified,  non-limiting  aperture,  this  tradeoff  is  expressible 
by  the  proportionality 

s  =  isu! 

where  S  is  the  sensitivity,  F#  is  the  system  F  number  and  Rd  is 
the  data  rate.  Since  practical  F#'s  are  constrained  to  1  £  F#  <  3, 
and  the  desire  for  commonality  of  equipment  dictates  TV  compatible 
data  rates,  this  tradeoff  is  very  limited  in  current  systems.  The 
result  is  designs  that  are  inflexible  with  respect  to  changing 
environmental  conditions. 

State-of-the-art  improvements  in  sensitivity  and  resolution  are 
expected  in  future  systems.  However,  there  are  theoretical  and 
practical  limitations  to  the  degree  of  improvement  that  can  be 
expected  (e.g.,  the  photon  noise  limits  of  thermal  detectors, 
physical  limitations  on  minumum  detector  size,  number  of  detectors, 
etc.).  Therefore,  it  is  almost  certain  that  the  sensitivity/reso¬ 
lution/data  rate  tradeoff  will  continue  to  influence  design  in 
the  future.  (See  Figure  17  for  a  graphic  presentation  of  the 
tradeoff  between  resolution  and  target  acquisition.) 

The  apparent  temperature  contrasts  of  typical  targets,  viewed 
against  a  terrestrial  background,  are  very  small.  The  correspond¬ 
ing  analog  voltage  contrasts  are  also  small,  so  that,  if  these 
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RESOLUTION  LIMITED  PROBABILITY 


Probability  of  Target  Acquisition  as 
of  FLIR  System  Resolution 


voltages  are  converted  linearly  into  luminance  contrast  on  a 
FLIR  visual  display,  the  operator  would  have  difficulty  recog¬ 
nizing  and  detecting  (Figure  16).  Atmospheric  effects  such,  as 
weather  changes,  humidity  and  haze  also  reduce  contrast  by  re¬ 
ducing  transmission  of  thermal  information  and  reducing  detection 
ranges  (see  Figure  18).  Contrast  commonly  is  increased  by  a-c 
coupling  the  detector  signal,  which  removes  the  low-frequency 
background  signal  and  improves  the  contrast  on  the  displayed 
image . 

Low-frequency  information  in  the  scene  will  be  distorted  or  lost 
by  a-c  coupling.  Image  defects  may  result.  In  some  cases,  this 
behavior  may  mask  other  targets.  Because  the  average  value  of  the 
a-c  coupling  circuit  is  zero,  the  large  positive  signal  response 
to  a  hot  target  will  be  followed  by  a  smaller,  but  longer  negative 
signal,  which  may  extend  all  the  way  across  the  display. 

Another  constraining  tradeoff  in  designing  thermal  imagers  is  the 
interdependence  of  sensitivity,  resolution,  and  depth  of  field. 
This  can  be  expressed  in  classical  geometric  optics  terms  as 


S  f '  6 


where  <*>  is  the  angular  radius  of  the  defocus  blue  circle,  S  is 
the  lens  speed,  f  is  the  focal  length,  6  is  the  depth  of  field 
and  r  is  the  range.  Thus,  high  resolution  (w)  and  sensitivity 
(high  lens  speed  S)  must  result  in  a  reduced  depth  of  field  (5). 
The  defocusing  parameter  A  is  measured  in  units  of  Rayleigh's 
(A/4)  tolerance  on  defocusing.  These  curves  demonstrate  that  the 
MTF  falls  off  very  rapidly  as  a  function  of  defocus  at  low  F*'s. 
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re  18.  Transmission  of  FLIR  Sensor  Information 
as  a  Function  of  Range 


Current  systems  use  dual  FOV  (field  of  view)  optics  to  overcome 
this  problem  to  some  extent.  Targets  are  acquired  with  a  wide 
FOV,  low-resolution  setting  which  has  less  of  a  depth  of  field 
problem,  and  then  a  narrow  FOV,  high-resolution  setting  can  be 
selected  by  the  operator  for  target  recognition  and  identification 
This  limits  search  effectiveness  at  long  range,  however. 

In  addition  to  depth-of-f ield  problems,  there  are  other  optical 
problems  that  affect  system  resolution.  Thermal  expansion  or 
contraction  of  optical  elements  (particularly  the  germanium 
lenses  used  for  thermal  imaging)  makes  fixed  focal  length  systems 
difficult  to  focus.  Also,  even  if  the  system  is  focused  on  the 
optical  axis,  the  optical  PSF  (point  spread  function)  is  not  con¬ 
stant  over  the  FOV. 


Second-Generation  FLIR  Configurations 

Advances  in  detector  and  CCD/CID  technologies  portend  large,  one¬ 
dimensional  or  two-dimensional  arrays  capable  of  multiplexing  and 
time-delay  integration  at  TV  rates.  Scan  configurations  that  are 
under  consideration  because  of  these  developments  include  series- 
parallel  configurations,  the  parallel  scan-pushbroom  array,  star¬ 
ing  arrays,  and  a  dithered  array  concept.  (Note  the  latter  is 
simply  a  hybrid  of  a  scanned  and  a  staring  array.) 

The  parallel  scan,  pushbroom-type  system  consists  of  one  or  more 
multi-element  arrays  positioned  horizontally  across  the  field  of 
view.  The  vertical  scan  is  mechanical,  and  the  horizontal  scan 
is  implemented  by  CCD  multiplexing.  Each  array  will  consist  of 
as  many  detector  elements  as  the  required  horizontal  resolution. 
D-c  coupling  and  background  subtraction  will  eliminate  the  need 


for  d-c  restoration.  Responsivity  equalization  will  be  attained 
through  an  AGC  amplifier  controlled  by  a  ROM,  or  from  calibration 
signals  derived  in  real  time.  Since  the  detectors  are  scanned 
vertically  and  sampled  once  each  horizontal  line,  less  bandwidth 
is  required,  higher  sensitivity  is  attainable,  and  imagery  corre¬ 
lations  in  all  directions  are  maintained.  Because  background  sub¬ 
traction  and  responsivity  equalization  will  be  accomplished  on- 
chip,  good  picture  uniformity  and  horizon  definition  should  re¬ 
sult.  This  type  of  scan  method  is  also  directly  TV-compatible. 

Now  being  explored  are  second-generation  FLIR  designs  incorpo¬ 
rating  improved  d-c  restore  techniques,  adaptive  contrast  enhance¬ 
ment,  and  automatic  gain  and  brightness  controls  (Figure  19)  which 
would  overcome  the  current  dynamic  range  limitations. 

Adaptive  or  selectable  intra-  and  inter-frame  averaging  will  re¬ 
duce  the  effect  of  the  sensitivity/resolution  design  constraint. 
Similarly,  adding  resolution  restoration  techniques  (e.g.,  super¬ 
resolution)  may  reduce  optical  aperture  size  requirements  at 
longer  wave-lengths,  or  augment  an  automatic  focus  control  func¬ 
tion  to  provide  uniform  resolution  over  the  entire  frame. 

Incorporating  these  image-processing  functions  into  a  second- 
generation  FLIR  design  effectively  requires  careful  attention  to 
the  interaction  between  the  various  functional  components  of  the 
FLIR,  such  as  the  interaction  between  the  detector  array/scan 
format  configuration  and  implementation  of  the  various  image 
processing  functions.  For  example,  initial  studies  of  super  reso¬ 
lution  algorithms  for  overcoming  the  Rayleigh  limit  quickly  iden¬ 
tified  the  need  for  oversampling  the  image  to  avoid  aliasing 
effects . 
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Video-image  processing  tradeoffs  and  selection  are  related  to  the 
scanning  techniques  chosen.  A  parallel  scanned  system  requires 
parallel  processing.  Thus,  the  number  of  electronic  channels  re¬ 
quired  is  equal  to  the  number  of  detector  elements  in  the  array. 

Two  basic  choices  are  possible  to  convert  the  video  signals  to  an 
optical  image.  One  is  by  electronic  multiplexing  using  CCDs  or 
CIDs  followed  by  conventional  TV  display.  The  other  approach  uses 
parallel  processing  to  an  LED  array  which  is  scanned  in  synchro¬ 
nism  with  the  detector  array  to  produce  the  display.  For  this 
approach,  the  controls,  such  as  gain,  level,  and  image  inversion 
must  be  applied  to  each  channel,  and  the  channels  must  track  each 
other  to  maintain  image  quality.  If  this  method  is  used,  the  dis¬ 
play  is  an  integral  part  of  the  FLIR  and  only  direct  viewing  is 
possible.  For  remote  viewing,  TV  camera  and  monitor  are  required 
and  add  further  complexity  to  the  system. 

A  basic  FLIR  sensor  consists  of  optics,  a  scanner,  detector  and 
cooler,  electronics,  and  display.  In  designing  a  FLIR,  it  is  neces¬ 
sary  to  account  also  for  the  characteristics  of  target  and  back¬ 
ground,  intervening  atmosphere,  and  the  human  operator.  Incorpor¬ 
ating  image-enhancement  techniques  into  FLIRs  will,  of  course, 
have  the  most  significant  effect  on  the  system's  electronics. 
Conversely,  processing  on  the  focal  plane  will  be  the  most  signi¬ 
ficant  constraint  on  image  enhancement . 

Focal  plane  processing  will  be  determined  by  CCD  and  CCD-compati¬ 
ble  technologies.  Background  subtraction  and  a  detector  array 
readout  using  CCD  multiplexing  are  already  near  reality.  CCD  or 
CID  delay  lines  will  provide  the  memory  to  implement  many  of  the 
enhancement  algorithms. 
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This  brief  discussion  of  potential  second-generation  FLIR  configu¬ 
rations  has  been  merely  to  focus  on  anticipated  future  systems. 

As  such,  the  discussion  is  not  intended  to  be  complete  but  merely 
to  reflect  current  thinking. 


FLIR  SUMMARY 

For  navigation  and  effective  target  surveillance  the  FLIR  scanner 
must  cover  40  to  60  degrees  across  the  horizon  and  20  to  30  degrees 
in  elevation.  This  coverage  is  approximately  equal  to  the  radar 
footprint.  For  detecting  small  ground  targets  such  as  trucks, 
tanks  and  missile  launchers,  approximately  1000  pixels  are  re¬ 
quired  in  the  horizontal  axis.  After  detection,  the  system  may 
zoom  or  change  optics  to  provide  a  1-1/2  to  3-degree  field  of  view 
for  target  recognition.  The  system  operates  in  the  8  to  12.5 
micron  region  with  60  fields  per  second. 

RADAR  SUMMARY 

A  number  of  points  about  radar  are  of  note: 

*  Angular  resolution  (antenna  size)  is  the  driving 
function,  which  forces  the  radar  designer  toward 
MMW  frequencies  to  keep  the  size  of  antenna 
apertures  reasonable . 

*  Terrain-following  (X  and  Ku  band)  radars  use 
relatively  large  antennas.  Some  airborne  MTI 
surveillance  systems  (SOTAS)  require  a  16  foot 
antenna  for  a  0.7  degree  beamwidth.  Therefore, 
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microwave  frequencies  (Ku  band  and  below)  are 
not  practically  suited  for  the  space  available 
in  a  small ,  high-performance  aircraft . 

*  Synthetic  Aperture  Radars  (SAR)  are  not  suited 
for  real-time  displays  due  to  processing  require 

ments . 

*  MMW  radars  operating  at  35  GHz  or  94  GHz  can 
provide  a  high  resolution  data  output.  The 
limiting  factor  on  maximum  detectable  range  is 
rainfall  attenuation.  However,  ranges  on  the 
order  of  3  -  7  km  can  be  expected  with  a  proba¬ 
bility  greater  than  0.99,  using  the  western 
Europe  weather  model . 

*  Passive  radiometers  operating  at  MMW  frequen¬ 
cies  typically  have  a  maximum  range  of  less 
than  1000  feet  for  conventional  targets. 

FLIR  AND  RADAR  SUMMARY 

Both  systems  have  several  points  of  comparison: 

*  Both  systems  have  reasonable  detection  ranges 
in  good  and  poor  weather. 

*  Both  systems  cover  an  adequate  footprint . 

*  The  systems  are  not  directly  compatible  due  to 
resolution  differences  (Radar  -  1  degree  beam 
and  IR  -  0.8  MR  pixel) 


*  The  size  of  the  system  is  such  that  it  will  fit 
in  small  attack  and  V/STOL  aircraft. 

Table  4  presents  estimated  target  detection  distances  for  a  2.7 
x  5.2  meters  target  using  a  3-watt,  35-GHz  radar  and  a  1000-pixel, 
40-degree  FOV  FLIR  system.  As  expected,  the  radar  has  almost  twice 
the  range  of  the  FLIR  system  under  clear  atmospheric  conditions. 
However,  4  mm/hr.  of  rain  attenuates  the  35-GHz  radar  by  38  per¬ 
cent  and  the  FLIR  by  only  25  percent .  The  16-GHz  radar  is  not 
significantly  attenuated  by  the  rainfall. 


TABLE  4.  ESTIMATED  RADAR  AND  IR 
DETECTION  DISTANCES 


> 

Radar 

FLIR 

35  GHz 

36  GHz 

1 °C  T  to  B* 

5°C  T  to  B* 

Clear  Atm. 

4.5  KM 

5  KM 

2.4  KM 

2.8  KM 

Rain  4  mm/hr. 

2.8  KM 

5  KM 

1.8  KM 

2.1  KM 

*  T  to  B  =  Target  to  Background 


SECTION  4 

SELECTED  MULTI SENSOR  DISPLAY  DESIGN  CONCEPTS 


Several  approaches  were  considered  for  combining  displays  and  how 
they  relate  to  mission  activities.  From  this  analysis,  two  new 
concepts  were  developed.  They  represent  a  new  method  of  preproces¬ 
sing  and  combing  the  two  sensor  outputs  on  a  single,  unified  dis¬ 
play.  The  two  methods  are  similar,  but  one  is  achromatic  and  the 
other  uses  color. 


BACKGROUND 

Many  of  the  direct  optical  and  electrical  combining  approaches  are 
listed  in  Figure  20.  However,  the  combined  display  must  offer 
benefits  when  applied  to  the  aircraft  and  its  various  missions. 

The  battlefield-interdiction  and  close-air-support  mission  are 
typical  applications  of  current  Navy  carrier  aircraft  and  forth¬ 
coming  VSTOL  aircraft. 

Discussions  with  pilots  and  an  analysis  of  time  lines  indicate 
high  workloads  and  a  high  probability  of  errors  during  the  pene¬ 
tration  and  attack  periods,  with  either  good  or  limited  visibility. 
The  low-altitude,  penetration  period  requires  information  for 
navigating  and  for  avoiding  terrain.  The  attack  period  requires 
data  for  target  detection,  recognition,  and  action  (e.g.,  a  deci¬ 
sion  about  which  weapon  to  use). 
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1.  IMAGES  FILLING  THE  WHOLE  SCREEN 

1  MV  BE  SHOWN  SIMULTANEOUSLY  ON 
MULTIPLE  SCREENS. 


IMAGES  FILLING  THE  WHOLE  SCREEN 
MAY  BE  SHOWN  SIMULTANEOUSLY, 
SUPERIMPOSED  UPON  EACH  OTHER  ON 

a  single  Screen 


3.1  IMAGES  FILLING  THE  WHOLE  SCREEN 
MAY  BE  SHOWN  SEPARATELY,  J1JXTA' 
I  POSED  IN  TIME.  ON  A  SINGLE 
sC'ftEE'fj - 


4. 


IMAGES  OCCUPYING  EQUAL  PROPOR¬ 
TIONS  OF  THE  DISPLAY  SPACE  MAY 
BE  SHOWN  SIMULTANEOUSLY  ON  A 
SINGLE  SCREEN.  THIS  IS  KNOWN 
AS  A  SPLIT-SCREEN  PRESENTATION. 
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A  FULL-SCREEN  IMAGE  MAY  8E 
SHOWN  WITH  A  DESIGNATED  PORTION 
OF  IT  OVERLAYED  BY  THE  SUPER¬ 
IMPOSED  INSET  OF  A  SECOND  IMAGE. 


A  FULL-SCREEN  IMAGE  MAY  BE  SHOWN 
WITH  AN  INSET  OF  A  SECOND 
IMAGE  REPLACING  A  DESIGNATED 
PORTION  OF  IT. 


7. 


A  VARIANT  ON  NUMBER  6,  WITH  THE 
INSET  IMAGE  PLACED  IN  ONE 
CORNER  OF  THE  MAIN  IMAGE,  AND 
A  MARKER  INDICATING  AN  ARE  * 

OF  SPECIAL  INTEREST  IN  THE 
LATTER. 


8. 


A  VARIANT  ON  NUMBER  7,  WITH  AN 
INSET  AGAIN  PLACED  IN  A  CORNER 
OF  THE  MAIN  IMAGE  IN  THIS  CASE, 
THERE  IS  NO  MARKER,  AS  THERE  IS 
NO  SPECIAL  RELATIONSHIP  BETWEEN 
THE  TWO  IMAGES. 


Figure  20.  Methods  of  Presenting  Multiple  Images 


With  high  visibility,  the  penetration  period  requires  only  the 
conventional  instruments  and,  possibly,  a  map  display.  However, 
with  limited  visibility,  the  pilot  needs  a  visual  presentation 
of  the  real  world.  This  is  best  achieved  by  an  imaging  sensor 
such  as  LLLTV  or  FLIR.  The  loss  in  resolution,  field  of  view, 
color  and  other  ground  cues  makes  distance  and  range  judgments 
more  difficult  for  terrain-avoidance  maneuvers.  An  imaging  dis¬ 
play  with  range  data  should  reduce  errors  in  judging  ranges. 

Detecting  ground  targets  under  both  limited  and  unlimited  visi¬ 
bility  conditions  are  enhanced  when  sensors  in  different  regions 
of  the  spectrum  search  for  target  signatures.  FLIR  and  LLLTV 
detect  heat  in  the  infrared  regions  (near  and  far).  Radar  detects 
reflections  and  the  scattering  of  high-frequency  radio  beams. 

Because  of  the  frequency  separation  of  the  sensors  (Figure  21), 
it  is  unlikely  that  a  target  could  escape  being  detected  by  both 
IR  and  radar,  unless  advanced  camouflage  techniques  are  employed. 

The  probability  is  also  very  high  that  the  military  target  being 
sought  on  the  mission  will  be  sensed  by  both  sensors  and  that 
false  targets  (such  as  large  animals  for  IR  and  rock  reflections 
for  radar)  will  be  sensed  by  only  one  of  the  two  sensors. 

On  many  current  missions,  two  target  passes  are  required,  the 
first  to  detect  and  verify  the  target  and  the  second  to  launch 
the  weapons.  The  addition  of  FLIR  and  radar  signatui  data  on 
the  same  imaging  display  used  for  navigation  may  allow  a  single 
pass  for  detecting,  verifying  launching  the  weapon. 
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Figure  21.  Atmospheric  Attenuation  versus  Frequency 

Victor  W.  Richard,  MMW  Radar  Applications  to  Weapon  Systems, 

BRL  Report,  Memo  Report  2631  Page  11,  1976. 


COMBINING  SENSOR  INPUTS 


The  problem  is  to  define  how  the  two  data  sets  could  be  meaning¬ 
fully  and  clearly  combined  in  one  simultaneously  presented  picture. 
Traditionally,  each  sensor  display  has  been  presented  separately 
to  the  operator  (multiple-display  format).  Some  consideration  has 
been  given  to  presenting  both  on  one  display,  thus  saving  panel 
space  in  the  cockpit.  Using  this  multifunction  display  concept, 
the  operator  would  switch  back  and  forth,  as  desired,  between 
FLIR  and  radar . 

Since  the  focus  of  this  study  was  to  go  beyond  such  formats,  the 
question  became  one  of  determining  how  to  present  both  sets  of 
sensor  data  simultaneously  on  one  surface. 

Superimposing  the  two  sensor  images  did  not  appear  to  be  a  feas¬ 
ible  alternative,  primarily  because  of  the  differences  in  signa¬ 
ture  information,  resolution,  and  display  format.  The  resulting 
combined  picture  would  almost  certainly  confuse  or  distract  the 
operator  rather  than  help. 

A  more  attractive  alternative  was  to  use  the  information  from 
one  sensor  to  annotate  the  image  presented  by  the  second.  One 
of  the  weaknesses  of  FLIR  is  the  lack  of  good  range  information. 

As  discussed  above,  one  weakness  of  the  radar  image  is  its  poor 
resolution.  Since  the  FLIR  image  provides  a  good  representation 
of  the  terrain,  it  appeared  to  be  the  most  logical  candidate  for 
the  primary  display.  Radar  data  could  then  be  used  to  supplement 
FLIR  data  by  adding  both  target  detection  and  ranging  informa¬ 
tion  . 


Image-processing  and  automatic-target -detection  algorithms  are 
being  actively  studied  by  a  number  of  investigators.  Currently 
available  hardware  and  software  provided  the  capability  for  auto¬ 
matically  scanning  sensor  output,  detecting  targets  of  interest, 
and  displaying  the  cue  on  the  operator's  display.  In  the  combined 
display  concept,  sensor  returns  from  both  FLIR  and  radar  would 
be  preprocessed.  The  displayed  FLIR  image  would  be  thus  annotated 
with  both  FLIR  and  radar  target  cues.  Using  different  cues  for 
each  sensor,  the  operator  would  quickly  determine  which  sensor 
had  picked  up  a  given  target.  A  third  cue  would  indicate  that 
both  FLIR  and  radar  had  detected  the  same  target. 

Adding  radar  range  information  to  the  FLIR  image  would  further 
enhance  its  utility.  The  method  of  providing  this  information 
(i.e.,  how  to  display  it)  is  another  issue.  Two  methods  are  dis¬ 
cussed  more  completely  below. 


SENSOR  IMAGE  PROCESSING 

Typically,  hot  objects  and  good  emitters  appear  on  IR  displays 
as  brighter  and  with  higher  contrast.  Highly  reflective  objects 
also  appear  on  the  radar  display  as  brighter  and  higher  contrast 
objects.  Therefore,  the  increase  in  signal  strength  above  a 
threshold  may  indicate  a  possible  target. 

After  1985,  it  is  assumed  that  the  sensor  imagery  will  be  prepro¬ 
cessed  by  automatic  target  screening  algorithms  controlled  by 
on-board  computers.  Each  sensor  output  will  be  automatically 
screened  prior  to  its  presentation  on  the  operator's  display. 
Potential  targets  detected  by  such  a  system  will  be  highlighted 
on  the  display  in  some  manner.  Currently  available  systems  detect 
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targets  at  a  fairly  high  level  of  probability.  Post -1985  systems 
will  perform  with  even  greater  accuracy.  Each  sensor  output  will 
have  its  own  preprocessor  and  its  own  associated  processing  algo¬ 
rithms.  Thus,  no  matter  what  the  final  display  mode,  it  will  be 
possible  to  separately  define  radar-cued  targets,  IR-cued  targets 
and  joint  IR-radar  cued  targets.  The  question  is  how  best  to  dis¬ 
play  this  cue  to  the  operator.  Two  important  considerations  about 
cue  selection  include: 

1)  The  cue  must  be  easily  seen  and  recognized  by 
the  sensor  operator. 

2)  The  cue  itself  should  not  obscure  either  part  or 
all  of  the  target,  nor  should  it  obscure  parts 
of  the  immediate  background.  The  background  it¬ 
self  may  contain  important  clues  useful  to  the 
operator  when  identifying  a  target. 

The  best  method  of  achieving  the  above  two  goals  appears  to  be 
through  highlighting.  The  target  and  its  immediate  surroundings 
could  be  cued  by  changing  the  luminance  and/or  the  dynamic  range 
of  the  locally  cued  area  relative  to  the  rest  of  the  displayed 
scene.  In  a  color  display,  the  cued  area  could  be  colored  distinc¬ 
tively,  making  it  highly  visible  to  the  operator.  Further  infor¬ 
mation  about  the  target  could  be  provided  by  changing  the  shape 
of  the  highlighted  area. 
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DISPLAYING  MULTIPLE  SENSOR  DATA 


Given  that  the  operator  has  two  or  more  sensors  onboard,  the  data 
can  be  displayed  in  a  variety  of  ways.  The  traditional  approach 
is  to  present  individual  sensor  data  on  separate  displays  (multi¬ 
display  approach).  Another  possibility  is  to  use  the  same  display 
surface  to  successively  present  one  or  the  other  sensor's  data 
(multifunction  approach). 

Each  of  these  display  alternatives  presents  problems  for  the  opera¬ 
tor.  In  either  the  multidisplay  or  multifunction  method,  he  may 
miss  important  information  on  one  display  while  looking  at  the 
other.  Additional  problems  include  the  necessity  of  constantly 
changing  orientation  from  one  form  of  data  to  another  very  dif¬ 
ferent  one.  That  is,  range,  resolution,  and  content  are  quite  dif¬ 
ferent  on  the  two  displays.  It  may  also  be  difficult  to  locate 
identical  points  on  the  two  displays  when,  for  example,  the  radar 
display  indicates  a  potential  target  which  the  operator  would 
like  to  examine  on  the  FLIR  display. 

The  general  combined-display  concept  selected  for  testing  attempts 
to  remedy  these  problems  by  presenting  both  sensor  returns  simul¬ 
taneously  on  a  single  display.  While  the  other  two  concepts  can 
be  functionally  considered  as  combined  display  concepts,  they  do 
present  the  above  problems.  Thus,  while  the  multi-display  might, 
in  fact,  be  a  single  split  screen  display,  it  does  not  truly 
combine  the  two  sensor  outputs  in  the  best  possible  way  from  the 
operator's  point  of  view. 
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The  most  useful  combined  display  would  take  the  information  from 
both  sensors  and  combine  it  prior  to  display.  The  alternative 
method  derived  in  this  study  uses  the  FLIR  image  as  the  basic 
operator  display.  The  FLIR  image  is  highlighted  and  modified  in 
the  following  ways.  Range  data  from  the  radar  is  added  to  the 
FLIR  display  such  that  the  operator  can  determine  relative  dis¬ 
tances  of  objects  on  the  FLIR  image.  Using  automatic  target  screen¬ 
ing  devices,  both  the  FLIR  and  radar  images  are  preprocessed.  Tar¬ 
gets  detected  by  either  FLIR  or  radar  are  highlighted  on  the 
display.  The  display  of  these  target  cues  is  shape-coded  such  that 
the  operator  can  determine  which  sensor  cue  is  being  displayed. 

If  both  sensors  have  picked  up  the  same  target,  the  area  would 
be  highlighted  using  a  third  shape  cue. 

Such  a  display  should  provide  the  operator  with  all  the  information 
required  on  a  single  display  surface  in  a  format  that  is  very  easy 
to  use.  It  is  assumed  that  the  operator's  scanning  time  would  be 
reduced,  and  target  acquisition  enhanced. 

The  above  display  could  further  be  presented  in  either  color  or 
black  and  white.  Figure  22  illustrates  this  combined  display  con¬ 
cept  in  an  achromatic  display.  Targets  are  highlighted  by  chang¬ 
ing  the  dynamic  range  of  the  immediate  target  area.  The  different 
shapes  represent  different  sensor  returns,  as  discussed  above. 

The  dashed  line  through  the  center  of  the  image  is  radar  range 
information . 

In  Figure  23,  the  same  image  presented  in  Figure  22  is  shown,  but 
with  the  addition  of  color.  In  this  image,  color  is  used  in  two 
ways.  The  cued  targets  are  highlighted  using  color  contrast  to 
make  them  more  visible.  The  different  sensor  returns  are  cued  by 
shape  in  the  same  manner  as  in  Figure  22. 
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Color  is  also  used  in  this  image  to  provide  radar  ranging  informa¬ 
tion.  Objects  at  close  range  are  colored  red,  those  at  intermediate 
range,  green,  and  those  at  a  far  range,  blue.  Thus  it  should  be 
possible  for  the  operator  to  see  color-cued  targets  and  to  estimate 
their  approximate  range.  Color  range  data  would  also  provide  a  navi¬ 
gational  cue  to  the  operator. 

While  the  features  of  the  combined  displays  illustrated  in  Figures 
22  and  23,  appear  to  provide  obvious  advantages  to  the  operator, 
particularly  in  a  single-seat  attack  aircraft,  they  have  not  been 
tested  experimentally.  During  the  current  contract,  a  test  plan  for 
evaluating  multisensor,  combined  displays  has  been  developed.  The 
test  plan  is  presented  in  Section  V.  The  actual  test  will  be 
conducted  in  Phase  III. 


SECTION  5 

EXPERIMENTAL  DESIGN 


Four  multiple  sensor  display  concepts  were  tested  using  dynamic 
imagery  with  various  targets  embedded  and  highlighted  cued  in 
selected  ways.  Two  of  these  display  concepts  combined  display 
formats  while  the  other  two  represented  uncombined  display  for¬ 
mats.  For  the  combined  display  formats,  the  FLIR  image  display 
was  the  basic  operator  display  presentation.  Radar  data  was 
then  combined  into  this  display,  supplementing  the  FLIR  data  by 
adding  both  target  detection  and  ranging  information.  The  two 
combined  display  formats  differed  in  that  one  was  achromatic 
while  the  other  used  color. 

The  two  uncombined  display  formats  presented  FLIR  and  Radar  in¬ 
formation  separately  in  conventional  displays.  Both  uncombined 
display  formats  were  achromatic.  These  display  formats  differed 
in  that  one  used  two  separate  but  simultaneous  displays  which 
employed  two  monitors  (multidisplays),  while  the  other  used  a 
single  display  monitor  (multifunction).  With  this  display  concept, 
the  operator  was  required  to  switch  back  and  forth,  as  desired, 
between  FLIR  and  radar  displays. 

In  each  experimental  condition,  the  imagery  displayed  on  the 
monitor(s)  simulated  the  penetration  phase  of  a  mission  the  low 
and  fast  approach  to  a  battlefield.  The  aircraft's  altitude  was 
approximately  200  feet  with  an  airspeed  of  200  knots.  The  low- 
altitude  flight  represented  an  aircraft  flying  into  enemy  ter¬ 
ritory  under  the  radar  net  to  avoid  being  detected.  The  imagery, 
as  viewed  by  the  observer,  contained  military,  and  false  targets. 


The  aircraft  with  automatic  target  screening  device,  automatically 
detected  targets  with  both  its  FLIR  and  radar  sensors.  A  total  of 
30  targets  were  embedded  in  the  imagery,  the  sequence  of  which 
lasted  approximately  five  minutes. 

The  30  targets  were  randomly  embedded  throughout  the  five-minute 
imagery  display.  Twenty-four  of  these  targets  were  highlighted, 
while  six  were  not.  The  targets  and  their  surrounding  terrain 
were  highlighted  by  increasing  each  image's  contrast  through  an 
expanded  dynamic  range.  Highlightings  were  shape-coded  to  signify 
the  specific  sensor  detection  device.  Circular  highlightings 
represented  detection  by  both  FLIR  and  radar  sensors.  Square 
highlightings  represented  detection  by  the  FLIR  sensor  only  and 
triangular  highlightings  represented  detection  by  the  radar  sen¬ 
sor  only.  Table  5  summarizes  the  target  characteristics  for  this 
imagery  display. 


TABLE  5.  HIGHLIGHTING  SHAPE  BY  SENSOR 


Target  Type 

Circle 
Both  Radar 
and  IR 

Square 

IR  Only 

Triangle 
Radar  Only 

Unhighlighted 

Undetected 

Tank  (Military) 

2 

2 

2 

2 

Helicopter 

2 

2 

2 

2 

Burner  (Non -Military) 

2 

2 

2 

2 

False  Target 

2 

2 

2 

- 

All  four  display  concepts,  in  addition  to  the  aforementioned  target 
detection  information,  afforded  the  operator  radar  ranging  informa¬ 
tion  as  well.  In  the  color,  combined-display  condition,  this  infor¬ 
mation  was  conveyed  through  the  color  bands  spanning  the  display 
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screen.  The  close  range  was  colored  red,  the  intermediate  range 
green,  and  the  far  range  blue.  In  the  black-and-white  combined 
display  condition,  radar  ranging  information  was  conveyed  by 
boundary  lines  superimposed  on  the  FLIR  display.  Uncombined  dis¬ 
play  conditions  revealed  radar  ranging  information  on  the  sectioned 
PPI  terrain  avoidance  radar,  displayed  on  a  separate  monitor.  In 
the  single-monitor  display,  a  switch  was  used  to  select  either  the 
radar  displays  or  the  FLIR  display,  while  in  the  two-monitor  dis¬ 
play,  the  radar-range  display  was  presented  concurrently  with,  but 
separate  from  the  FLIR- image  display. 

The  partial  PPI  radar  also  displayed  signatures.  Helicopter 
signatures  appeared  as  horizontal  oblongs  (— ) ,  tanks 
as  circles  (•),  and  burners  as  vertical  oblongs  (|).  Highlighted 
targets  were  all  presented  with  a  circular  ring  surrounding  the 
target  (Q) .  The  absence  of  a  ring  represented  an  unhighlighted 
target.  Consequently,  the  uncombined  display  detected  targets 
from  both  the  radar  and  FLIR  displays.  While  the  combined  display 
conditions  only  afforded  the  subject  a  single  source  of  target 
detection  information,  this  display  combined  the  information  from 
both  sensing  devices  onto  the  single-screen  display.  The  differ¬ 
ential  information,  then,  was  affected  by  the  number  and  kind  of 
highlighted  targets  presented. 


In  summary,  then,  each  subject  performed  three  tasks:  1)  "point- 
of-impact,"  2)  target  detection,  and  3)  tracking. 

All  four  display  conditions  annotated  the  imagery  display  with  a 
cross  hair,  which  showed  "point -of -impact”  information  to  the 
subject.  The  uncombined  condition  displays  showed  the  impact  point 
on  the  partial  PPI  terrain-avoidance  radar  display.  The  subject 
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responded  to  this  information  by  indicating  the  range  zone  at 
which  this  projected  point  of  impact  was  occurring.  If  the  cross 
hair  was  in  the  near  range  zone,  the  observer's  task  was  to  pull 
back  a  three-position  lever  with  his  left  hand;  for  the  far-range 
zone  positioning,  the  lever  was  pushed  forward.  An  intermediate 
range  zone  was  indicated  by  placing  the  level  in  a  center  position 
Lever  adjustments  were  not  uniformly  distributed  throughout  the 
five-minute  session,  consequently  lever  adjustments  were  neces¬ 
sary  anywhere  from  every  1  to  13  seconds.  The  changes  were  dic¬ 
tated  by  the  terrain.  The  range  zones  changed  50  times  during 
each  experimental  run. 

The  above  described  "point-of-impact"  task  was  the  subject's 
primary  task  in  each  experiment .  Two  other  concurrent  tasks  were 
required  of  each  subject  for  every  experimental  condition. 

The  secondary  task  of  this  experiment  was  that  of  "target  detec¬ 
tion."  For  this,  subjects  were  required  to  pronounce  the  name  of 
each  target  type  (helicopter,  tank,  burner  or  false  target)  as 
soon  as  each  was  recognized  and  detected. 

The  third  and  least  significant  task  in  this  experiment  was  one 
of  tracking  throughout  the  five-minute  run.  A  right-handed,  spring 
loaded  joystick  was  employed  to  control  a  randomly  moving  dot 
about  a  prescribed  area.  The  subjects'  task  was  to  attempt  to  keep 
the  dot  inside  a  circle  embedded  within  this  area.  The  tracking 
task  appeared  superimposed  on  the  combined  and  FLIR  display 
for  each  subject  across  all  conditions. 


Thirty-two  right-handed  male  subjects  were  used.  The  subject's 
ages  ranged  from  19  to  43,  with  the  made  being  in  the  mid-20s. 

Each  subject  participated  in  two  experimental  conditions:  one  com¬ 
bined  and  one  uncombined.  Half  of  the  subjects  viewed  the  color 
combined  display,  while  the  other  half  viewed  the  black-and- 
white  combined  display  for  their  combined  condition  experimental 
trial.  Similarly,  half  of  the  subjects  viewed  a  single-monitor 
display,  while  the  other  half  viewed  a  double -monitor  display  for 
their  uncombined  condition  experimental  trial.  Half  of  the  sub¬ 
jects  first  participated  in  a  combined  condition  and  then  an 
uncombined  condition,  while  the  other  half  received  the  opposite 
treatment:  first  uncombined  and  then  combined.  Consequently, 
four  subjects  were  randomly  assigned  to  each  of  the  eight  cells 
possible.  The  following  listing  displays  the  design: 


8  Subjects 

8  Subjects 

8  Subjects 

8  Subjects 

Color 

Color 

Black  & 
White 

Black  & 
White 

Combined 

One  math- 

function 

monitor 

2  Monitors 
one-FLIR 
one -Radar 

One  multi¬ 
function 
monitor 

2  Monitors 

one-FLIR 

one-IR 

Uncombined 

All  subjects  were  first  exposed  to  a  training  session  (varying  with 
condition:  see  training  procedures  section)  then  an  experimental 
condition.  Following  a  short  break,  subjects  were  then  exposed  to 
a  second  training  session,  followed  by  a  second  experimental  con¬ 
dition  . 

An  entire  session  (i.e.,  two  experimental  conditions  plus  two 
training  sessions)  for  each  subject  lasted  approximately  two  hours. 
Subjects  were  paid  $15  for  their  participation. 


SECTION  6 

EXPERIMENTAL  STIMULUS  GENERATION  PROCEDURE 


A  five-minute  film  simulating  the  cockpit  view  from  a  low  and  fast 
approaching  aircraft  into  a  battlefield  was  needed  as  the  major 
stimulus  material  for  this  study.  Due  to  the  nature  of  the  target 
detection  and  point-of-impact  data  needed  to  make  this  film,  a 
complex  series  of  procedures  were  developed  to  produce  a  finished 
product . 

To  simulate  the  pilot's  perspective  from  the  cockpit  of  a  low- 
flying,  high-speed  aircraft,  the  initial  task  was  to  film  a 
dynamic  aerial  view  over  an  expanse  of  land.  This  was  to  serve 
as  the  background  terrain  for  the  needed  imagery  display.  A 
Hiller  UH-12E  helicopter  was  equiped  with  two  cameras:  a  Tele- 
mation  1100  camera  (with  a  red-extended  vidicon)  and  a  high- 
resolution  16  mm  Arrow  Flex  Model  S.  Both  were  secured  to  shock 
mounts  to  reduce  vibrations.  A  portable  tape  recorder  on  the 
helicopter  transformed  the  video  imagery  onto  video  tapes.  Al¬ 
though  two  types  of  imagery  film  were  collected  (red  extended 
vidicon  and  high-resolution  16  mm  films)  the  imagery  filed  ex¬ 
clusively  by  the  Telemation  1100  camera  with  red-extended  vidi¬ 
con  was  used  for  our  experimental  imagery.  This  decision  was 
made  by  Commander  Donald  Hanson  as  he  judged  it  to  be  a  closer 
approximation  of  IR  imagery  simulation.  The  helicopter, 
flying  at  approximately  60  knots,  covered  areas  of  Minnesota's 
north  shore,  flying  low  in  a  generally  northeasterly  direction 
from  the  shoreline  through  the  valleys  and  surrounding  rolling 
hills.  The  steep,  tree-covered  hills  presented  an  excellent 
navigation  problem  and  the  clearings  in  the  valley  offered  many 
potential  target  sites. 
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The  video  tapes  from  the  helicopter  flight  were  then  refined  in 
several  ways.  The  film  first  needed  to  be  edited  to  more  closely 
approximate  the  high-speed  (200  knots)  of  the  low-flying  aircraft 
intended  for  simulation.  The  film  also  needed  to  be  prepared  so 
that  frame-by-frame  target  detection  and  point-of-impact  data 
annotations  could  be  subsequently  performed.  This  was  done  by 
transforming  the  video  tapes  onto  video  discs  using  an  Arvin 
Echo  video  disc  recorder /player . 

Initially,  every  tenth  frame  of  the  original  video  tape  was  re¬ 
corded  onto  the  video  disc.  That  is,  three  frames  per  second  of 
the  original  film  (at  30  frames  per  second)  were  now  recorded. 
After  processing  each  frame  a  new  tape  was  developed  which  con¬ 
tained  a  series  of  3  identical  frames  (^  (60)^200).  This  process 
transformed  the  video  tape  to  reflect  the  air  speed  of  a  high¬ 
speed  (200  knots)  military  aircraft. 

The  next  step  in  this  procedure  called  for  frame-by-frame  annota¬ 
tions  of  point-of-impact  and  target  detection  data.  Point-of- 
impact  information  was  needed  for  every  single  frame  in  the 
entire  film  segment,  target  detection  data  annotations  to  the 
film  were  needed  only  for  a  total  of  600  frames.  The  entire 
film  segment  contained  30  embedded  targets,  and  each  target 
appeared  for  two  seconds  of  real  time.  Therefore,  20  different 
frames  per  target  were  used  for  annotation.  Targets  were  em¬ 
bedded  randomly  throughout  the  film  segment.  Their  spot  appear¬ 
ances  in  the  film  segment  were  dictated  exclusively  by  the 
nature  of  the  filmed  terrain.  Targets  were  generally  embedded  in 
clearings.  Such  areas  facilitated  developing  a  realistic  film 
segment  in  that  targets  could  be  embedded  in  appropriate  places 
from  frame  to  frame,  thus  appearing  stationary.  Their  movement 
across  the  screen  only  reflected  the  aircraft's  movement  as  it 
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approached,  and  eventually  flew  over  them.  Such  movement,  then, 
reflected  only  changes  in  distance. 

Once  the  entire  imagery  tape  was  properly  edited  and  recorded 
onto  the  video  discs,  a  frame-by-frame  procedure  for  incorporat¬ 
ing  the  range  detection  data  was  instituted  by  using  a  Stanford 
Technology  Corporation  Model  70-E  image  computer.  The  video  signal 
of  each  frame  was  digitized  and  stored  in  the  memory  of  the  Model 
70-E.  With  a  trackball  and  cursor,  two  boundary  lines  (which 
divided  the  imagery  display  into  three  regions)  were  then  drawn 
on  the  video  monitor  projecting  the  imagery  frames.  These  boundary 
lines  were  stored  in  the  graphics  memory  of  the  computer.  The 
execution  of  the  computer  program  overlayed  the  line  with  the 
digitized  imagery. 

While  the  shapes  and  angles  of  the  boundary  lines  were  determined 
by  the  shape  and  angle  of  the  horizon  as  projected  in  each  imagery 
frame,  the  distance  between  the  boundary  lines  was  dictated  by 
the  nature  of  the  terrain  projected  in  each  imagery  frame. 

Very  flat  terrain  dictated  widely  spaced  boundary  lines  with 
little  terrain  below  the  lower  boundary  line,  mountainous  terrain 
was  reflected  by  greater  distances  between  the  bottom  of  the 
screen  and  the  lower  boundary  line  and,  consequently,  smaller 
distances  between  the  upper  and  lower  boundary  lines. 

The  overlayed  boundary  lines  on  the  digitized  imagery  frame  were 
employed  differently  in  two  distinctly  different  functions  as 
executed  by  the  computer  program.  In  one  condition,  white  dashed 
lines  were  actually  superimposed  onto  a  black-and-white  imagery 
frame  resulting  in  an  imagery  display  sectioned  into  three  parts 
(see  Figure  22). 


The  second  function  used  the  boundary  lines  drawn  with  the  track¬ 
ball  and  cursor  as  delimiters  for  the  three  sections,  which  were 
then  filled  in  with  colors.  That  is,  the  region  between  the  bot¬ 
tom  of  the  screen  and  the  lower  boundary  line  was  colored  red, 
the  region  between  the  two  boundary  lines  green,  and  the  region 
between  the  upper  boundary  line  and  the  horizon  blue.  Consequent¬ 
ly,  in  this,  the  nonachromatic  representation  of  each  imagery 
frame,  near  distances  to  the  aircraft  were  colored  red,  middle 
distances  green,  and  far  distances  blue  (see  Figure  23) . 

A  third  function  of  the  Model  70-E  program  projected  each  digi¬ 
tized  imagery  frame  without  the  overlayed  boundaries.  Consequently, 
this  entire  procedure  resulted  in  three  distinctly  different  end 
products  for  each  imagery  frame,  each  to  be  employed  as  stimulus 
material  for  different  experimental  conditions.  (See  Experimen¬ 
tal  Design  Section  for  details.)  One  presented  the  imagery  frame 
in  black  and  white  with  no  boundary  lines  overlaid.  This  was  the 
uncombined  FLIR  conditions.  Another  presented  the  imagrey  frame 
in  black  and  white  with  two  white  dashed  lines  superimposed  upon 
it.  This  was  the  black  and  white  combined  display,  the  third 
presented  a  color-coded  imagery  display,  with  each  of  the  three 
regions  reflected  by  a  different  and  distinct  color.  This  was  the 
color-combined  display. 

A  cross  hair,  centrally  located  on  each  imagery  frame,  was  also 
embedded.  This  cross  hair,  combined  with  the  boundary  lines  over¬ 
lay,  offered  the  subject  all  the  needed  information  for  the 
"point -of-impact"  task.  (See  Experimental  Design  Section  for 
details . ) 
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As  mentioned  earlier,  some  imagery  framed  needed  target  detection 
information,  as  well  as  point -of-impact  information.  For  these 
imagery  frames,  additional  procedures  similar  to  the  ones  just 
outlined  were  necessary.  Initially,  black-and-white  slides  of  all 
pertinent  targets  (helicopter,  tank,  burner)  had  to  be  developed. 
These  slides  had  to  reflect  each  target  as  detected  from  varying 
distances.  Consequently,  multiple  slides  of  each  target,  each 
slide  reflecting  a  different  sized  target,  were  processed  using 
a  Pentax  KX  camera.  A  series  of  20  photographs  were  taken  at 
successive  distances  from  the  target. 

With  a  Telemation  100  camera  and  a  mast  127S  Random  Access  Pro¬ 
jector,  each  slide  was  digitized  and  stored  in  memory.  The 
location  of  the  to-be-embedded  target  was  then  set  by  the  cursor 
indicator  and  the  threshold  for  the  video  image  was  set  by  the 
trackball.  After  this  needed  information  was  entered  into  the 
Model  70-E,  the  program  was  able  to  embed  the  target  in  the 
background  imagery  in  each  selected  video  frame.  The  program 
also  allowed  for  one  of  four  highlighting  options,  based  upon 
sensor  detector:  1)  circle  (both),  2)  triangle  (radar),  3)  square 
(FLIR),  4)  no  highlighting  (no  sensor).  Highlighting  options 
were  determined  randomly  as  each  target  appeared  twice  in  each 
highlighting  condition  option  (see  Experimental  Design  section 
for  detail . ) 

The  highlighted  area  of  interest  underwent  a  computer  operation 
to  enhance  the  target.  The  area  was  expanded  linearity  to  the 
maximum  limits  of  the  dynamic  range.  The  whites  become  whiter 
and  the  blacks  blacker.  On  the  color  concept,  the  area  was  coded 
in  yellow.  A  triangle  represented  the  detection  of  the  radar,  a 
square  by  the  IR,  and  a  circle  by  both.  The  single  combined  dis- 


plays  contained  the  three  highlights.  The  uncombined  FLIR  dis¬ 
play  contained  only  highlights  sensed  by  the  IR  system.  The  radar 
contained  highlights  sensed  by  the  radar  system.  After  specify¬ 
ing  a  location  for  a  given  target,  20  successive  frames  under¬ 
went  this  procedure.  The  efforts  of  this  procedure  resulted  in 
the  appearance  of  a  stationary  ground  target  whose  movement  and 
size  changes  reflected  the  aircraft's  movement  approaching  and 
eventually  bypassing  the  specific  target  location.  Such  procedures 
were  followed  for  all  30  targets. 

Although  the  description  of  the  boundary  lines  overlay  preceded 
the  description  of  the  target  overlay  in  this  report  due  to  the 
greater  frequency  of  use  of  the  former,  the  Model  70-E  program 
executed  target-embedding  first,  and  boundary  lines  overlaying 
second.  For  the  majority  of  frames  in  which  boundary  lines  over¬ 
lays,  were  necessary  in  the  absence  of  target  overlay,  target 
embedding  was  skipped  by  simply  directing  the  program  to  its 
second  execution  only. 

The  process  of  annotating  frames  with  the  necessary  ranging  and 
target  detection  information  continued  until  all  30  targets  were 
embedded  properly.  An  elapsed  real  time  of  four  seconds  (i.e., 

40  frames)  was  dictated  as  the  shortest  interval  between  the 
offset  of  one  target  and  the  onset  of  the  next  one.  This  entire 
procedure  resulted  in  a  film  of  four  minutes  56  seconds.  A  total 
of  34  200-frame  video  discs  of  background  imagery  were  used. 

The  resulting  imagery  with  specified  annotations  as  embedded  by 
the  Model  70-E  image  computer  and  displayed  onto  the  Tektronix 
650  A-l  monitor  was  recorded  on  video  cassettes  by  a  Sony  U- 
Matic  video  cassette  recording  unit.  Because  of  the  image  pro¬ 
cessing  computer  program  in  the  Model  70-E,  the  three  different 
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types  of  annotations  (see  above)  for  each  frame  were  recorded 
in  succession.  This  necessitated  a  selective  rerecording  of  the 
video  cassettes  back  to  video  discs  so  that  successive  recordings 
of  similarly  annotated  frames  could  be  grouped.  That  is,  a  selec¬ 
tive  editing  process  of  the  video  cassettes  to  video  discs  re¬ 
sulted  in  three  different  tape  segments  with  sequential  and 
similarly  appearing  (i.e.,  same  type  of  annotation)  frames. 

The  discs  were  recorded  back  to  video  cassettes  for  final  edit¬ 
ing  to  eradicate  all  imagery  overlap  and  produce  a  final  product 
continuous  flight  simulation  film  segments. 

The  editing  facilities  used  were  at  Gillette  Children's  Hospital, 
St.  Paul,  Minnesota:  Two  Sony  2860  video  cassette  recorders,  a 
Z-6  Video  Media  mixing  control  unit ,  a  Telemedia  36-10A  char¬ 
acter  generator,  and  a  Panasonic  WJ  200  series  video  switcher. 

This  process  resulted  in  three  five-minute  film  segments,  one 
for  each  Model  70-E  function  described  above.  The  black-and- 
white  film  segment  with  dashed  lines  was  employed  in  the  black- 
and-white  combined  experimental  condition,  and  the  color  band 
imagery  was  employed  in  the  color-combined  experimental  condition. 
No  further  imagery  were  then  needed  for  these  experimental  con¬ 
ditions.  However,  this  was  not  the  case  for  the  remaining  imagery 
display  developed.  The  black-and-white  imagery  with  no  boundary 
lines  was  employed  in  both  the  single-  and  double-monitor  uncom¬ 
bined  conditions  but  not  exclusively.  This  imagery,  with  the 
absence  of  range-finding  data,  yielding  only  target  detection 
data,  represented  IR  simulation  only.  Consequently,  an  additional 
imagery  display  simulating  a  radar  display  needed  yet  to  be  de¬ 
veloped  for  the  uncombined  conditions.  The  procedure  for  develop¬ 
ing  this  final  imagery  display  follows. 
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The  radar  imagery  display  developed  for  use  in  uncombined  condi¬ 
tions,  was  distinctively  different  than  those  described  earlier. 
Consequently,  an  entirely  different  developmental  process  was 
employed.  The  radar-display  imagery,  unlike  the  realistic-looking 
aerial  photography  imagery,  was  symbolic  in  nature.  A  simulated 
sectioned  PPI  radar  pattern  was  used  as  the  background  imagery 
for  this  display  (see  Figure  24). 

The  display  sheets  had  the  radar  grid  and  were  correlated  with 
the  aerial  video  tapes  and  were  updated  every  third  frame.  Point- 
of-impact  information  (i.e.,  appropriately  placing  a  cross  hair 
in  one  of  the  three  sections  of  the  display)  and  target-detection 
information  (i.e.,  symbolic  representations  of  each  target  with 
appropriate  highlighting  annotations)  were  hand  drawn  on  each 
sheet.  These  hand-drawn  annotations  reflected  the  identical  in¬ 
formation  on  the  parallel  IR  simulation  display.  The  two  displays 
differed  only  in  their  mode  of  presentation  (symbolic  versus 
realistic).  Approximately  300  sheets  were  developed. 

With  the  use  of  the  Telemation  1100  camera,  each  radar  simulation 
sheet  was  projected  onto  a  Tektronix  650  A-l  monitor.  Each  pro¬ 
jected  image  was  then  recorded  onto  the  Arvin  Echo  video  disc 
player/recorder .  These  video  disc  recordings  were  then  transformed 
onto  video  cassettes  to  prepare  the  imagery  for  the  final  editing 
process.  This  process  employed  the  Gillette  Children's  Hospital 
editing  facilities  as  described  earlier. 

The  last  procedure  was  to  attach  a  leader  tape  to  the  beginning 
of  both  the  IR  simulation  display  tape  (black  and  white  with  no 
boundary  overlays),  and  to  the  radar  simulation  display  tape. 

The  leader  tape  countdown  assured  that  both  tapes  were  run  exactly 
simultaneously  in  the  uncombined  conditions  for  which  they  were 
eventually  used. 
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Figure  24.  Simulated  Radar  as  Background  Imagery  iry 


SECTION  7 

TRAINING  PROCEDURE 


All  subjects  were  trained  for  a  period  prior  to  each  experimental 
condition.  Since  each  subject  participated  in  two  experimental 
conditions,  two  training  sessions  were  given  each  subject.  While 
the  second  training  period  lasted  approximately  15  minutes,  the 
first  extended  for  up  to  45  minutes.  The  more  brief  second  train¬ 
ing  session  was  largely  due  to  the  subject's  familiarity  with  the 
work,  gathered  from  the  similar  nature  of  the  tasks  already  com¬ 
pleted  in  the  first  experimental  condition.  In  fact,  the  tasks  in 
both  experimental  sessions  were  identical,  only  the  conditions  for 
them  changed. 

Subjects  were  expected  to  perform  three  tasks  concurrently  during 
each  experimental  condition.  The  tasks  were  ranked  according  to 
their  significance:  1)  point  of  impact,  2)  target  detection,  and 
3)  tracking  (See  Experimental  Procedure  for  Detail).  Subjects  were 
trained  for  these  tasks  in  the  reverse  order  of  their  significance, 
to  this  experiment.  That  is,  subjects  were  first  trained  for  the 
tracking  task,  then  the  target  detection  task,  and  finally  the 
point  of  impact  task.  Certain  aspects  of  the  training  session 
varied  from  subject  to  subject,  depending  upon  the  experimental 
conditions  involved  and  the  order  in  which  they  were  to  be  con¬ 
ducted.  Other  aspects  of  training  remained  constant  across  all 
subjects  at  all  times. 

The  first  part  of  the  training  session  addressed  itself  to  the 
tracking  task,  the  least  significant  task.  Across  all  subjects 
for  all  experimental  conditions,  this  training  segment  remained 
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uniform.  All  subjects,  prior  to  the  first  experimental  condition, 
were  given  five  one-minute  training  trials  at  the  task.  In  all 
but  three  instances,  subjects  stabilized  their  accuracy  by  the 
end  of  the  fifth  trial.  For  those  three  subjects  who  continued 
to  better  their  performance  through  all  five  trials,  two  addi¬ 
tional  one-minute  training  trials  were  added.  All  three  subjects 
then  displayed  performance  stability. 

The  tracking  task  remained  unchanged  in  the  second  experimental 
condition  across  all  subjects  as  well.  Training  for  the  tracking 
task  prior  to  the  second  experimental  condition  consisted  of 
two  one-minute  trials  at  the  tracking  task.  All  but  four  subjects 
reached  their  baseline  performance  level  within  this  alotted 
time.  The  four  subjects  who  continued  to  better  their  tracking 
performances  were  given  a  single  additional  one-minute  training 
trial.  A  baseline  performance  level  was  then  reached  by  each  of 
these  subjects. 


Following  the  training  trials  preceding  both  the  first  and 
second  experimental  conditions,  subjects  were  given  the  oppor¬ 
tunity  to  question  any  areas  of  uncertainty  concerning  the  track¬ 
ing  task.  The  training  session  continued  on  the  second  task, 
target  detection,  only  when  each  subject  acknowledged  a  complete 
understanding  of  the  tracking  task. 

The  second  part  of  the  training  session  addressed  the  task  of 
target  detection.  This  training  component  was  formatted  consist¬ 
ently  across  all  subjects,  but  varied  in  the  stimulus  material 
presented,  depending  on  the  experimental  condition  being  addressed. 
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All  subjects  performed  in  both  a  combined  and  an  uncombined  con¬ 
dition  (see  experimental  procedures  for  detail).  Combined  condi¬ 
tions  were  either  in  color  or  black  and  white.  Uncombined  condi¬ 
tions  were  either  single-  and  double-monitor  displayed.  These 
variables  determined  the  nature  of  the  material  presented  during 
this,  the  second  part  of  the  training  session. 

All  subjects  were  initially  exposed  to  a  15-second  video  clip  of 
the  dynamic  nature  of  the  five-minute  video  tape  they  were  to  see 
during  the  experimental  condition.  Prior  to  viewing  this  video 
clip,  they  were  instructed  to  attend  to  the  specific  features  of 
the  imagery  altitude,  speed,  and  terrain  relevant  to  their  inte¬ 
grating  with  their  tasks  during  the  experiment.  Subjects  in  the 
color  condition  viewed  a  colored  film  segment,  while  subjects  in 
the  black-and-white  conditions  viewed  black-and-white  film. 

All  subjects  across  all  conditions  were  then  exposed  to  still 
photographs  of  each  of  the  possible  targets  to  be  detected.  These 
displays  exposed  the  subject  to  the  largest  possible  target  size 
that  would  ever  appear  during  the  experimental  condition.  That 

is,  although  the  subject  would  be  exposed  to  smaller  pictures  of 
each  of  the  targets  during  the  experimental  condition,  he  would 
never  see  larger  sizes  of  the  targets  that  those  shown  during 
the  training  period.  Targets  were  then  presented  to  the  subject 
in  highlighted  conditions.  Although  not  all  target -by -highlighting 
combinations  were  shown,  a  representative  sample  of  these  combi¬ 
nations  were  viewed  and  an  explanation  of  the  remaining  possible 
target -by-highlighting  types  presented.  Training  did  not  continue 
until  the  subject  expressed  a  complete  understanding  of  the  pos¬ 
sible  target  shapes  and  highlighting  combinations  that  he  may 
have  had  to  view  during  the  experimental  condition.  For  subjects 
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in  the  color  condition,  all  targets  and  highlightings  were  dis¬ 
played  in  color.  For  subjects  in  black-and-white  conditions,  all 
displays  were  in  black  and  white . 

Prior  to  the  showing  of  this  video  clip,  subjects  were  asked  to 
attend  to  specific  features  of  the  display  containing  pertinent 
information  for  their  experimental  condition,  duration  time  of 
targets  on  the  screen,  changing  sizes  of  targets,  constant  size 
of  highlighting  shapes,  contrast,  and  color.  Once  again,  subjects 
in  the  color  condition  viewed  a  color-video  clip  while  subjects 
in  black  and  white  condition  viewed  a  black  and  white  video  clip. 

After  the  video  clip  presentation,  any  questions  on  the  task  were 
answered  and  a  second,  similar  video  clip  was  shown.  Subjects  were 
now  asked  to  respond  by  identifying  the  targets  by  simply  calling 
out  their  names  just  as  they  would  be  required  to  do  during  the 
experimental  condition.  Feedback  (correctness  or  incorrectness  of 
response)  was  given  to  the  subject  after  each  response.  Again, 
color-condition  subjects  viewed  a  color  video  clip  and  black  and 
white-condition  subjects  viewed  a  black  and  white  video  clip. 

When  subjects  were  run  under  the  combined  conditions,  the  second 
section  of  the  training  period  ended.  When  they  were  run  uncom¬ 
bined  conditions,  an  additional  training  period  was  needed  to 
explain  the  added  (radar )display  and  its  function  for  the  target- 
detection  task. 

All  subjects  were  run  in  both  combined  and  uncombined  conditions. 
However,  some  subjects  did  a  combined  conditions  first  and  an 
uncombined  condition  second  while  others  performed  in  the  oppo¬ 
site  order.  Consequently,  all  subjects  were  exposed  to  the  addi¬ 
tional  (radar)  display  training  period,  however,  for  some,  this 
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occurred  in  the  training  period  preceeding  the  first  conditions, 
while  for  others  this  occurred  in  the  training  period  preceeding 
the  second  experimental  condition. 

The  additional  training  period  needed  for  the  target -detection 
task  in  the  uncombined  conditions  consisted  of  familiarizing  the 
subjects  with  the  radar  signature  for  the  targets,  both  high¬ 
lighted  and  unhighlighted,  and  the  background  against  which  they 
were  to  be  dynamically  displayed.  Subsequently,  a  small  video 
clip  was  shown  integrating  both  target  movement  and  a  dynamic 
background  display.  The  subjects  were  instructed  that,  during 
the  experimental  condition,  this  display  would  be  used  concurrently 
with  the  one  previously  viewed.  Similarities  and  differences  in 
information  delivered  from  the  two  monitors  were  elaborated  upon. 
After  the  subjects  understood  the  nature  of  "the  dual-monitor  task 
and  the  added  instructions  necessary  for  it,  there  was  a  brief 
practice  trial.  Viewing  the  second  (radar)  display  only,  subjects 
responded  to  a  small  video  clip  by  calling  out  targets  just  as 
they  would  be  asked  to  do  during  the  experimental  condition.  Feed¬ 
back  (correctness  of  response)  was  given  and  all  subsequent  ques¬ 
tions  answered.  This  concluded,  the  extended  target  detection 
training  offered  to  all  subjects  when  run  in  the  uncombined  con¬ 
ditions  only. 

The  final  section  of  the  training  session  addressed  the  point -of- 
impact  task,  the  most  important  one.  Subjects  in  uncombined  con¬ 
ditions  received  different  training  that  subjects  in  the  combined 
conditions.  Since  all  subjects  ran  in  both  combined  and  uncombined 
conditions,  all  subjects  were  exposed  to  both  training  sections. 
Subjects  were  exposed  to  one  during  the  training  session  preceed¬ 
ing  the  first  experimental  condition,  and  to  the  other  in  the 
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training  session  preceeding  the  second  experimental  condition. 

The  order  in  which  subjects  experienced  training  depended  upon 
the  order  in  which  they  were  run. 

Training  for  the  target-detection  task  only  varied  from  the  com¬ 
bined  to  the  uncombined  condition  in  that  the  information  needed 
for  the  task  was  differentially  presented  in  the  different  monitor 
displays.  The  training,  however,  was  similar. 

For  both  training  sections,  the  experimenter  explained  the  pur¬ 
pose  and  function  of  the  task  and  how  each  condition  displayed 
the  needed  information.  Subject  responses  to  the  task  were  ex¬ 
plained  and  training  trials  followed.  Questions  were  answered 
until  the  experimenter  perceived  each  subject  completely  under¬ 
stood  the  task.  At  that  time,  an  intergrated  practice  task  (point 
of  impact  and  target  detection)  was  administered.  Subjects  re¬ 
sponded  to  the  trial  task  in  the  same  way  as  in  the  experimental 
condition.  Subsequently,  feedback  (performance,  accuracy  accounts) 
was  given  to  each  subject.  Again,  subjects  were  then  permitted 
to  ask  any  questions  concerning  all  three  tasks.  When  each  ques¬ 
tion  was  adequately  answered  and  the  experimenter  perceived  the 
subject  to  the  total  comprehension  of  his  experimental  condition 
tasks,  the  training  session  ended. 
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SECTION  8 

CONTROL  OF  THE  EXPERIMENT 


The  experiment  was  conducted  in  the  man /machine  laboratories 
under  the  control  of  an  Eclipse  Data  General  Computer.  During 
the  development  of  the  imagery,  a  code  was  added  across  the  bot¬ 
tom  of  each  frame  to  indicate  the  contents  of  that  frame.  The 
code  consisted  of  10  small  squares,  which  were  either  black  or 
white.  The  10  squares  represented  the  10  possible  conditions: 
points  of  impact  (near,  intermediate  or  far);  types  of  targets 
(helicopter,  task,  burner,  or  none),  and  highlighted  conditions 
(square,  triangle,  circle  or  none). 

Ten  photocells  were  placed  across  the  bottom  of  the  CRT.  The 
computer  read  the  status  of  these  cells  (on  or  off),  the  position 
of  the  point  of  impact  switch  (near,  intermediate  or  far)  and 
the  position  of  the  joystick  (X  and  Y) .  In  addition,  a  voice-recog¬ 
nition  system  recorded  target  recognition  responses  from  the  sub¬ 
jects,  and  provided  the  computer  with  the  time  when  the  utterance 
began  and  the  type  of  target  detected. 

The  codes  on  the  tape  allow  response  time  to  be  measured  within 
two  video  frames,  or  approximately  66  milliseconds.  The  capstan 
driven  video  tape  recorder (s),  one  for  combined  and  two  for  un¬ 
combined  displays,  were  driven  by  a  common  time  base  corrector 
and  synchronized  within  one  frame.  The  recorders  were  keyed  and 
started  on  timing  frames  ahead  of  the  actual  imagery.  At  the  end 
of  a  run,  a  high-speed  printer  listed  the  times  of  the  changes 
and  responses. 
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The  two-dimensional  tracking  task  was  generated  in  a  Nova  Data 
General  computer  and  displayed  on  a  stroke  writing  Megateck  dis¬ 
play.  A  high-resolution  video  camera  focusing  on  the  stroke 
writing  display,  converted  the  tracking  task  to  a  525-TV-line 
raster  format.  A  video  mixer  inserted  the  tracking  task  in  the 
upper  center  of  combined  and  IR  displays.  The  white-tracking  task 
on  the  darker  imagery  produced  a  high  contrast .  The  computer  also 
calculated  RMS  tracking  errors. 

A  wood  mockup  was  fabricated  to  hold  the  displays  and  controls. 

A  12-inch  high-resolution  color  monitor  at  eye  level  presented 
both  combined-display  conditions,  multifunction  conditions  and 
FLIR  uncombined  data.  A  13-inch  black-and-white  monitor  just 
below  the  color  monitor  presented  the  radar  data  for  the  multi¬ 
display  condition.  A  trigger  switch  operated  by  the  index  finger 
on  the  tracking  stick  allowed  the  operator  to  select  either  the 
FLIR  or  radar  display  during  the  multifunction  conditions. 


SECTION  9 
RESULTS 


Four  performance  measures  were  recorded  for  each  subject's  display 
conditions : 

1.  Tracking  score  indicating  the  subject's  proportion 
of  single-task  performance  that  was  retained  under 
multiple  task  (tracking,  detecting,  impact  moni¬ 
toring  and  switching)  conditions .  This  was  a  number 
between  0.0  and  1.0,  based  on  measured  RMS  perform¬ 
ance  in  the  two  conditions. 

2.  Target  Detection  Time  indicating  time  from  onset 
of  target  to  vocal  response  identification . 

3.  Correct  Identifications  indicating  the  number  of 
targets  correctly  identified  under  the  trial.  This 
was  a  number  out  of  32  possible  targets. 

4.  Point-of-Impact  Shifts  indicating  the  number  of 
correct  changes  in  the  toggle  position  to  align 
with  the  cursor  position.  This  was  a  number  out 
of  50  possible  changes. 

These  variables  were  analyzed  several  ways  across  several  group¬ 
ings  in  the  experimental  design.  Recalling  our  design  with  four 
major  conditions: 
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Condition 

1 

2 

3 

4 


Sensor  Description 
Color  Combined 
Uncombined ,  one  monitor 
Black  and  White,  Combined 
Uncombined,  two  monitors 


We  see  that  conditions  (1)  and  (3)  are  "combined"  displays,  and 
that  (2)  and  (4)  are  uncombined  displays.  In  the  following  analy¬ 
ses,  these  conditions  were  compared  and  contrasted  in  several  ways. 
The  analyses  included: 


•  Correlational  analysis  of  all  dependent  variables 

•  Combined  vs  uncombined  display  comparisons 

-  Unvariate  analysis 

-  Grouping  of  subject  scores 

-  Multivariate  analysis 

•  Display  condition  comparisons 

•  Color  vs.  black-and-white  comparisons 

•  Sensor  detections/highlight  comparisons 

The  following  section  describe  these  analyses  and  results. 


CORRELATIONAL  ANALYSES 

To  assess  the  degree  of  relationship  between  our  four  performance 
measures,  correlation  scores  were  obtained  across  all  subjects. 
The  three  tables  are  shown  in  Table  6.  They  are  divided  into  com¬ 
bined  conditions  (1)  and  (3)  and  uncombined  conditions  (2)  and 
(4)  and  total  across  all  four  conditions.  The  correlations  are 
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TABLE  6.  CORRELATION  COEFFICIENTS 


TR 

RT 

CR 

POI 

TR 

- 

0.30664 

-0.09906 

0.06072 

RT 

- 

- 

0.29043 

0.13648 

CR 

- 

- 

- 

0.08956 

POI 

- 

- 

- 

- 

TR  RT  CR  POI 


TR 

- 

0.16648 

-0.19513 

-0.26782 

RT 

- 

0.00082 

0.11845 

CR 

- 

- 

0.17402 

POI 

- 

- 

— 

- 

TR 

RT 

CR 

POI 

TR 

0.23281 

-0.14476 

-0.10191 

RT 

- 

" 

0.14331 

0.12544 

Combined 


Un combined 


Total 


generally  low  and  below  the  level  of  significance  at  the  p  <  0.05 
level  (r  =  0.296).  One  correlation,  between  tracking  and  response 
time,  was  above  this  level  (0.307),  indicating  a  weak,  positive 
relationship  between  these  measures.  Generally,  the  measures  re¬ 
corded  showed  little  interrelationship  indicating  a  measurement 
of  different  human  capabilities, 

COMBINED  VERSUS  UNCOMBINED  DISPLAY  COMPARISONS 

Unvariate  Analysis  —  The  means  for  the  four  performance  measures 
across  combined  and  uncombined  conditions  are  presented  in  Figure 
25.  Standard  deviations  are  shown  in  brackets.  The  confidence 
level  is  shown  in  the  upper  right  corner  of  each  graph.  All  four 
measures  showed  statistically  significant  differences  between  the 
two  condition  groups.1  Tracking  performance,  correct  identifica¬ 
tions,  and  point-of-impact  changes  were  all  superior  under  com¬ 
bined  display  condition.  However,  an  analysis  of  the  stimulus 
material  indicates  a  possible  explanation  for  this  apparent  con¬ 
tradictory  effect .  The  FPI  radar  image  allowed  early  target  recog¬ 
nition  reducing  time  to  respond.  This  observation  indicates  that 
highlighting  the  area  of  interest  on  the  primary  display  is  not 
sufficient  and  that  the  combined  display  must  contain  inserts  of 
other  sensor  data  from  secondary  sensors. 

The  other  three  measures  indicated  the  expected  results.  Subjects 
had  more  residual  attention  for  tracking  combined  displays.  Com¬ 
bined  displays  enhanced  the  ability  of  the  subjects  to  identify 
targets  correctly. 

1  A  dependent  or  paired-data  t-test  was  performed  to  assess  the 
condition  differences  with  df=31. 
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Figure  25.  Combined  versus  Uncombined  Graphs  and  Confidence  Levels 


Subject  Data  Analysis  by  Grouping  —  The  subject's  performance  oi. 
all  four  measures  was  ranked.  From  this  rank  ordering,  two  groups 
were  established:  one  with  the  highest  scores,  and  one  with  the 
lowest  scores.  Each  group  contained  approximately  20  scores.  How¬ 
ever,  ties  did  not  permit  exact  cutoff  points.  This  analysis,  as 
shown  in  Table  7,  indicates  that  two  out  of  three  high-performance 
scores  on  tracking  and  correct  target  identification  occurred  in 
the  combined  condition.  It  also  indicates  by  the  same  margin  that 
poor  performance  occurred  during  the  uncombined  condition.  The 
response  time  measure  also  supported  the  contention  that  the  radar 
sensor  data  provided  earlier  target  recognition.  The  point -of-im- 
pact  analysis  indicated  a  much  less  pronounced  difference  between 
combined  and  uncombined  conditions.  During  the  training  period, 
the  subjects  were  instructed  to  give  the  highest  priority  to  the 
point-of-impact  tasks  (this  task  was  related  to  terrain  avoidance 
and  aircraft  altitudes).  This  instruction,  if  followed,  would  pro¬ 
duce  small  differences  between  high-priority  tasks  and  larger 
differences  between  the  low-priority  and  secondary  or  "workload" 
tasks . 

Multivariate  Analysis  —  To  provide  a  more  accurate  description 
of  the  performance  of  subjects  in  a  multi-dependent  variable 
situation  such  as  this  study,  one  should  consider  combining 
scores  in  a  linear  fashion  to  maximize  group  differences  and 
assess  the  contribution  of  each  variable  to  this  difference. 
Multivariate  discrimination  analyses  allows  us  to  combine  data 
from  several  dependent  scores  and  create  a  new  variable  which 
can  be  regarded  as  "performance."  The  data  from  this  experiment 
was  analyzed  to  maximize  the  group  differences  between  combined 
and  uncombined  display  conditions. 


TABLE  7.  COMBINED  VS.  UNCOMBINED  HIGH/LOW  GROUPS 


TRACKING 


LOW  GROUP  HIGH  GROUP 


Combined 

3 

15 % 

Combined 

14 

70% 

Uncombined 

17 

8o% 

Uncombined 

6 

30% 

Total 

20 

Total 

20 

CORRECT 

TARGET 

IDENTIFICATIONS 

LOW  GROUP 

HIGH  GROUP 

Combined 

7 

32% 

Combined 

15 

68% 

Uncombined 

15 

68% 

Uncombined 

7 

32% 

Total 

22 

Total 

22 

TIME 

TO  IDENTIFY  TARGETS 

LOW  GROUP 

HIGH  GROUP 

Combined 

13 

65% 

Combined 

3 

15% 

Uncombined 

7 

35% 

Uncombined 

17 

85% 

Total 

20 

Total 

20 

POINT  OF  IMPACT 

LOW  GROUP 

HIGH  GROUP 

Combined 

9 

41% 

Combined 

10 

50% 

Uncombined 

13 

59% 

Uncombined 

10 

50% 

Total 

22 

Total 

20 

Figure  26  is  a  composite  of  standard  score  data  distributions  for 
all  four  performance  measures  and  our  discriminate  scale , "perform¬ 
ance."  The  plots  show  that,  although  the  individual  measures  indi¬ 
cate  a  difference  in  distributions  for  combined  vs.  uncombined 
displays,  the  multivariate  analysis  markedly  increases  the  dis¬ 
tance  between  distributions. 

The  major  product  in  the  discriminate  analysis  is  a  set  of  weights 
indicating  the  relative  sensitivity  of  the  dependent  measures  to 
group  differences.  These  weights  indicate  the  best  linear  combina¬ 
tion  that  accentuates  the  differences  of  the  uncombined  groups. 

The  analysis  of  our  data  revealed  the  following  equation  for 
combined  data: 

Performance  *  0.67  TR  +  0.46  RT  +  0.48  Cl  +  0.32  PI 

where  TR  is  tracking,  RT  is  response  time.  Cl  is  correct  identifi¬ 
cation  and  PI  is  point-of -impact  data.  (The  function  was  signi¬ 
ficant  at  the  p  <  0.05  level.) 

The  function  shows  tracking  to  be  the  most  sensitive  to  display 
condition  changes,  followed  by  RT  and  Cl,  and  finally  PI.  This 
progression  corresponds  to  the  subject's  priorities  between  tasks, 
and  to  the  instructions.  PI  was  to  be  regarded  as  more  important 
than  either  target  recognition/detection  or  tracking.  Subjects 
were  able  to  devote  less  attention  to  tracking  in  the  uncombined 
display  conditions,  i.e.,  demonstrating  less  "reserve"  capacity 
for  a  low-priority  task.  Combined  displays  allowed  a  good  per¬ 
formance  on  even  the  lowest* priority  task,  while  maintaining  or 
increasing  the  performance  on  other  tasks. 
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DISPLAY  CONDITION  COMPARISONS 


Comparisons  of  Combined  and  Uncombined  Display  Conditions  —  The 
four  display  conditions  were  compared  in  pairs  by  t-tests  (df«7) 
of  the  various  measures.  These  data  are  presented  graphically  in 
Figure  27.  The  levels  of  significance  of  each  comparison  are 
shown  by  the  *'s  indicating  the  0.05,  0.01,  and  0.001  levels. 

Tracking  scores  were  significantly  better  in  all  comparisons, 
except  for  the  Color  Combined-vs . -Multifunction  Monitor  display 
comparison,  and  these  conditions  showed  differences  in  the  ex¬ 
pected  direction.  Tracking  was  consistently  better  across  the 
combined  display  conditions. 

Target  response  times  were  significantly  lower  in  only  two  con¬ 
dition  comparisons:  Color  vs.  Combined  FLIR  &  RADAR  Uncombined, 
and  Color  Combined  vs.  Multifunction  Uncombined.  The  other  two 
conditions  (Black-and-White  Combined  vs.  both  Uncombined  condi¬ 
tions)  showed  only  small  differences. 

Target's  identified  were  significantly  more  in  the  Color  Combined 
vs.  Multifunction,  and  Black-and-White  Combined  vs.  Multifunction 
display  comparisons.  The  other  conditional  comparisons  revealed 
only  small  differences. 

Point  of  impact  changes  were  significantly  higher  in  only  one 
comparison:  Color  Combined  vs.  Multifunction  Uncombined. 

Although  some  comparisons  were  not  significant,  most  were  in  the 
expected  direction. 


PERCENT  OF  TARGET  DETECTED  PERCENT  OT  SINGLE  TASK  RETAINED 
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Figure  27.  Combined  versus  Uncombined  Display  Conditions 
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COMPARISON  OF  COLOR  AND  BLACK-AND-WHITE  DISPLAYS 

The  two  experimental  conditions  combining  sensor  information  were 
compared  in  this  analysis.  This  was  a  comparison  of  the  color 
combined  vs.  black-and-white  combined  displays  (Conditions  1  vs,  3) 

An  independent  groups  t-test  (df=30)  was  performed  on  all  four 
performance  measures.  None  of  the  tests  showed  statistical  relia¬ 
bility  at  £  <  0.05  level,  although  number  of  targets  correctly 
identified  and  target  response  times  showed  trends  favoring  the 
black-and-white  system. 


SENSOR  DETECTIONS/HIGHLIGHT  COMPARISONS 

An  analysis  compared  the  combined/ uncombined  detection  perform¬ 
ances  of  the  three  highlight  methods: 

1)  Radar- only  highlight 

2)  FLIR-only  highlight 

3)  Both  FLIR  and  radar  highlight 

Data  was  collapsed  across  the  three  target  types.  Means  for  the 
combined  and  uncombined  conditions  are  presented  in  Table  8. 

Radar-only  Highlight  —  Our  analysis  of  the  radar  highlight 
method  showed  significantly  faster  target  response  times  and  a 
trend  toward  fewer  correct  identifications  for  the  uncombined 
displays.  Times  were  shorter  in  uncombined  displays  because  the 
radar  display  used  to  highlight  on  uncombined,  provided  target 
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TABLE  8.  TARGET  DETECTION  CONDITIONS 


TARGET  RESPONSE  TIMES  (SECONDS) 


Target 

Detected 

By 

Mean 

Combined 

Mean 

Uncombined 

Confidence 

Level 

Radar 

Only 

(A) 

1.89 

1.54 

P<0.001 

FLIR 

Only 
(  □  ) 

1.96 

1.82 

Both  Radar 
&  FLIR 
(0) 

1.93 

1.69 

P<0.001 

NUMBER  OF  TARGETS  CORRECTLY  RECOGNIZED 
(out  of  6  possible  targets) 


Target 

Detected 

By 

Mean 

Combined 

Mean 

Uncombined 

Confidence 

Level 

Radar 

Only 

(A) 

5.19 

4.66 

P  =  0.10 

FLIR 

Only 
(  □) 

5.44 

5.13 

P  >  0.10 

Both  Radar 
&  FLIR 
(0) 

5.50 

5.34 

P  >  0.10 
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information  sooner  than  on  the  combined  display.  Radar-only  high¬ 
lighted  targets  were  more  correctly  identified  on  the  combined 
display,  i.e.,  target  signature  was  more  defined  on  these  displays. 

FLIR-only  Highlight  —  The  FLIR-only  detected  targets  tended  to 
be  recognized  earlier  on  the  uncombined  displays,  although  this 
result  was  not  significant.  A  possible  explanation  for  the  trend 
is  that  after  the  subjects  found  a  highlighted  FLIR  target  on  the 
uncombined  FLIR  monitor,  they  jumped  to  the  uncombined  radar  moni¬ 
tor  for  a  rapid  confirmation.  In  this  condition,  approximately  the 
same  number  of  targets  were  correctly  recognized  in  the  combined 
and  uncombined  modes. 

FLIR  and  Radar  Highlight  —  When  both  sensors  detected  a  target, 
the  recognition  times  were  significantly  shorter  in  the  uncombined 
condition.  This  evidence  continues  to  support  the  contention  that 
the  subjects  were  using  the  radar  display  for  early  recognition. 
Again,  an  equal  number  of  targets  were  recognized  in  both  combined 
and  uncombined  conditions. 


SUMMARY 

This  data  indicates  that  combined  displays  must  present  the  most 
appropriate  target  display  information  for  each  short  increment 
of  time.  The  data  further  suggests  that  the  operators  may  desire 
to  switch  between  sensors  IR,  radar  and  millimeter  wave. 

In  addition,  handoff  and  automatic  selection  of  the  sensors  to  be 
displayed  must  be  developed.  That  is,  the  radar  is  best  at  long 
ranges,  however,  as  the  range  is  reduced,  the  FLIR  or  other  sensors 
become  more  appropriate.  The  difficult  area  is  in  the  transition 
when  more  than  one  sensor  is  providing  useful  data. 
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SECTION  10 


RECOMMEND AT IONS 


In  addition  to  conducting  the  laboratory  tests,  the  multisensor 
concepts  were  presented  to  both  Navy  and  Air  Force  pilots  for 
comments  and  criticism.  The  concepts  were  also  evaluated  in  re¬ 
lation  to  display  content  and  usefulness  on  close-air-support 
missions.  Therefore,  the  following  recommendations  are  based  on 
user  comments,  experimental  data,  and  analytical  data. 

1.  The  combined  display  must  attempt  to  present  the 
world  as  viewed  from  the  cockpit.  Abstract  dis¬ 
plays  which  do  not  relate  to  the  real  world,  or 
which  require  interpretation  create  confusion 
when  pilots  use  the  display  and  intermittently 
fly  VFR. 

2.  Keep  it  simple  -  Display  only  the  high-priority, 
essential  information.  Nonessential  data  only 
masks  the  key  information. 

3.  The  pure  addition  or  overlay  of  sensor  data 
(IR,  LLTV,  Millimeter  wave  and  radar)  does  not 
enhance,  but  degrades  the  targets.  The  direct 
overlay  approach  tends  to  obscure  the  critical 
data  in  each  display. 

4.  Use  a  black-and-white  display  to  transmit  high- 
frequency  information  or  imagery.  Pseudo-color 
tends  to  produce  slower  response  times. 
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5.  Color  should  be  used  to  reinforce  symbology, 
not  as  the  primary  method  of  coding.  That  is, 
if  color  were  not  present,  no  data  would  be 
lost  from  the  display. 

6.  One  display  format  will  not  serve  for  naviga¬ 
tion,  pop-up  and  attack.  The  mission  require¬ 
ments  change,  therefore,  the  optimum  display 
format  also  changes. 

7.  The  combined  display  must  present  the  most 
appropriate  sensor  information.  One  display 
highlighting  by  other  sensor  is  not  sufficient 

8.  The  display  must  provide  a  smooth  transistion 
as  the  mission  changes,  navigation  to  pop-up 
to  attack  and  also  as  the  appropriateness  of 
the  sensors  change,  radar  to  millimeter  wave 
to  FLIR  to  LLTV . 

9.  The  display  should  indicate  which  sensors  are 
above  a  threshold  for  the  area  of  interest. 

10.  The  display  should  emphasize  threat-radar 
trackers  at  missile  sites. 

11.  The  combining  of  the  sensor  should  not  use 
symbols  which  obscure  critical  information. 
Cross  hairs  may  block  most  of  a  target. 


The  combined  multisensor  displays  studied  and  evaluated  were 
preliminary  concepts.  The  following  two  advanced  concepts  were 
developed  by  using  the  11  preceding  recommendations.  Figure  28 
is  a  display  concept  for  a  high-speed,  low-altitude  penetration 
segment  of  the  mission.  Figure  29  shows  the  same  display  con¬ 
figured  for  the  pop-up  segment .  The  displays  have  the  following 
features : 

•  Range  Zones  -  The  near  zone  is  indicated  by  a  series 
of  small  red  and  white  circles  and  the  far  zone  by 
green  and  white  dashes.  If  the  colors  were  not 
present,  the  zones  line  are  still  distinguishable 

by  the  shape  (circles  and  dashes). 

•  Background  -  To  maintain  the  real-world  relationship, 
the  sensor  that  best  visually  represents  the  real 
world  is  used  as  the  background  for  the  display. 

•  Target  Highlight  -  When  the  output  of  a  sensor  exceeds 
a  threshold,  the  sensor  data  is  inserted  on  the  dis¬ 
play.  These  squares,  with  an  increased  dynamic  gray 
scale,  contain  the  actual  sensor  data,  not  the  high¬ 
lighting  of  the  background  sensor.  If  more  than  one 
sensor  exceeds  the  threshold  level,  the  box  will 
contain  the  image  from  the  sensor  highest  above  the 
threshold  level.  Color  bar(s)  on  the  sides  of  the 

box  indicate  which  sensor  detected  the  area.  Again, 
if  the  color  is  lost,  the  bar(s)  location  left,  right, 
top  or  bottom  still  indicate  the  sensor(s).  Also,  the 
bar  does  not  obstruct  the  contents  of  the  square. 
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Figure  29.  Advanced  Display  Concept  with 
Narrow  Field-of-View  Insert 


•  Threat  Sensors  -  These  areas  of  strong  emissions  are 
highlighted  by  a  red  circle.  The  circle  is  used  only 
to  indicate  threats. 

•  FOV  -  A  wide  FOV  is  required  to  fly  the  aircraft,  and 
the  narrow  high-resolution  FOV  is  required  for  target 
related  activities.  The  system  contains  high  resolu¬ 
tion  sensors  which  are  covering  the  area  within  the  ^  ^ 
cornered  rectangle.  This  rectangle  is  centered  on  an 
area  that  exceeds  a  threshold.  If  more  than  one  area 
exceeds  a  threshold,  the  cornered  rectangle  jumps 
around  the  threshold  areas  until  commanded  to  stop 

by  the  pilot.  At  this  time,  the  area  becomes  magnified 
about  its  center  point  (Figure  29).  Targets  often 
come  in  clusters,  therefore,  the  larger  area  within 
the  covered  rectangle  was  used  rather  than  the  smaller 
threshold  area. 
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